False negative
A false negative usually comes down to a test that doesn't actually check what it claims to. An assertion that's too lenient, confirming a page loaded rather than checking that the specific data on it is correct. A test that covers the happy path but never exercises the exact edge case where the defect actually lives. A mocked dependency that doesn't accurately reflect how the real thing behaves, so the test passes against a version of reality that isn't true. Or a test that's quietly broken in a way that always reports success, like an assertion wrapped in error handling that silently swallows a failure.
Of the two kinds of mistake, this is generally the more dangerous. A false positive wastes time, but it gets noticed, someone investigates a red result eventually. A false negative gives a green result, which means nobody investigates anything, the defect ships, and it surfaces later, in production, often found by a user instead of a test, which is a far more expensive place to discover it. A suite full of false positives at least signals that something's wrong, a false negative signals nothing at all.
Reducing them comes down to writing assertions specific enough to actually catch the defect they're meant to catch, not just confirming something ran without crashing, deliberately testing edge cases and failure conditions rather than only the happy path, reviewing test code with the same scrutiny as production code, since a bug in a test is just as real as a bug in the application, and periodically introducing a known bug on purpose to confirm the suite actually catches it, rather than assuming it would.
Common causes
- An assertion that's too lenient, checking that something ran without crashing rather than checking the specific result that actually matters.
- A test that only covers the happy path, never exercising the exact edge case where a defect actually lives.
- A mocked dependency that doesn't accurately reflect how the real thing behaves, letting the test pass against a version of reality that isn't true.
- A test that's quietly broken in a way that always reports success, like an assertion wrapped in error handling that silently swallows a failure.
Where it shows up
- A regression suite passing cleanly while a real defect ships straight to production, discovered later by a user instead of a test.
- A test that checks a page loaded, but never checks that the specific data displayed on it is actually correct.
- A mocked API response that doesn't match what the real service actually returns once it's live.
- A team trusting a green suite that was never actually exercising the code path where the bug lives.
Risks and mitigations
| Risk | Mitigation |
|---|---|
| A real defect ships to production because the test meant to catch it passed anyway, discovered later by a user instead. | Writing assertions specific enough to actually catch the defect, not just confirming something ran without crashing. |
| A team trusts a green suite that was never actually exercising the code path where the real bug lives. | Deliberately testing edge cases and failure conditions, not only the obvious happy path. |
| A mocked dependency passes a test against a version of reality that doesn't match what's actually true in production. | Periodically confirming a mock still reflects the real dependency's actual behavior, not just what it looked like when the mock was written. |
Frequently asked questions
What's the difference between a false negative and a false positive?
A false positive is a test that fails even though nothing's actually wrong, wasting time investigating a non-issue. A false negative is a test that passes even though something genuinely is wrong, letting a real defect through undetected, which is generally considered the more dangerous of the two since it goes unnoticed.
How can a team tell if its suite has false negatives?
It's hard to know directly, since by definition a false negative looks like a passing test. One common technique is mutation testing, deliberately introducing a small, known bug into the code and confirming the suite actually catches it, rather than assuming it would.
Why are false negatives considered more dangerous than false positives?
A false positive at least gets noticed, someone investigates a red result eventually. A false negative produces a green result, so nothing prompts anyone to look closer, and the real defect ships, often surfacing later in production where it's far more expensive to find and fix.
How Klarent helps
Klarent's AI agents reason about whether an application's actual behavior matches what a generated test expects, rather than relying on a shallow assertion that technically passes without really checking the thing that matters.