QAM Hub QAM Hub
Home / Glossary / Test coverage

Test coverage

Last reviewed

How much of something is exercised by tests, where the something has to be named to mean anything: requirements, features, code, risk. Requirement coverage is what test management systems measure, and it answers whether anything shipped untested. It is not code coverage, and quoting one number while meaning the other is the most common reporting error in QA.

Also called: requirement coverage

Name the denominator

"80% coverage" means nothing until you say 80% of what. Of requirements with at least one linked case? Of code lines executed by unit tests? Of manual cases that have an automated counterpart? Each of those is a legitimate number, each answers a different question, and they routinely differ by tens of points on the same product.

Requirement coverage versus code coverage

Code coverage comes from instrumenting the code and tells developers which lines or branches tests executed. Requirement coverage comes from links between requirements and test cases and tells the team whether every agreed behaviour has a test, and whether that test passed on the current build. A product can have high code coverage and an untested requirement, because unit tests exercised the code without checking the behaviour the customer asked for.

Covered is not the same as passing

A requirement with a linked case that last ran three months ago, or that failed yesterday, still counts as covered in a naive report. Useful coverage reporting shows three things side by side: linked, executed recently, passed. The gap between the first and the last is the honest number.

What coverage cannot tell you

Coverage measures presence, not quality. One weak case per requirement gives 100% and catches nothing. Read coverage next to defect escapes and the content of the cases, never on its own.

In QAM Hub

Requirement Coverage and Automation Coverage are reports available on every plan; Automation Coverage separates cases marked automated from cases an automated result actually linked to. See automation coverage reporting and the automation coverage term.

Back to the test management glossary