Test coverage is a vanity metric.
Escape rate isn't.
You can hit 90% coverage and still ship the same bugs. Why escape rate (defects that reach production) is the QA number that matters.
I've walked into more than one engagement where a team was proud of a 90-something percent coverage number and still shipping the same category of production bug, month after month. If coverage was that high, why were the same bugs still getting through? Coverage was never lying to them. It was just answering a question nobody actually needed answered.
01 · What coverage actually measures
Coverage tells you what percentage of your code executed while your tests ran. It says nothing, structurally cannot say anything, about whether the assertions that ran alongside that execution would have noticed if the result was wrong. A test can walk through every line of a function and check nothing meaningful at the end. That test counts fully toward coverage. It has caught exactly zero bugs, and it never will.
$ report --metric=coverage,escape-rate --range=q3test coverage ................. 93%escape rate (bugs reaching prod) 17 / monthtrend, 6 months ............... flat# 93% and flat. the number that should move isn't moving.
“A coverage report tells you where your tests went. It doesn't tell you whether they were looking at anything while they were there.”
02 · The test that passes without looking
The weak pattern is almost always the same shape, and once you've seen it you start noticing it everywhere: an assertion checks that a response exists, or that its status code is defined, instead of checking that it equals the specific value the business logic actually promises. expect(response.status).toBeDefined() will pass whether the endpoint returns a 200 or a 500. It executes the line. It detects nothing.
I've fixed entire suites by doing nothing but replacing loose existence checks with specific value assertions. Coverage percentage didn't move a point. Real defect detection did.
03 · Escape rate: the scoreboard that can't be gamed
Escape rate, the count of defects that reach production versus the count caught before release, ties QA directly to the outcome anyone actually cares about. You cannot inflate it by writing more tests that don't check anything, the way you can inflate coverage. It only moves when real bugs stop reaching real users, which is the entire point of having a QA function in the first place.
- 01Tag every production bug by where it should have been caught. Unit, integration, or end-to-end. This alone tells you which layer of your suite is actually failing you.
- 02Trend escape rate monthly, not coverage. Put it on the same dashboard leadership already looks at. A flat or rising trend next to a high coverage number is the conversation that changes priorities.
- 03Audit your highest-coverage files first, not your lowest. Counterintuitively, a file with 100% coverage and a history of escaped bugs usually has the weakest assertions in the codebase. It's stopped looking suspicious to anyone.
- 04Make "does this assertion check a specific value" part of code review. The cheapest fix is catching the weak pattern before it merges, not after it's one of a thousand tests nobody wants to touch.
// the team with the flat escape rate above wasn't short on tests. they had thousands. what they were short on was tests that would fail if the code were actually wrong.
04 · What to tell your leadership instead of a coverage number
Coverage is easy to report and easy to feel good about, which is exactly why it survives as the default QA metric in so many organizations. Escape rate is harder to report and impossible to fake, and it's the only one of the two that tells you whether your testing effort is actually protecting users. Report both if you have to. Make decisions based on the second one.
$ end of file · written by flux/dk · 2026-04-13