Home / Glossary

Test Management Glossary

Last updated 2026-10-16 · 31 terms

This is a glossary of test management terminology: the words QA teams use to organise, execute and report on testing inside a test management system. It is deliberately not a general software testing dictionary. You will not find black box testing or performance testing here, because those are well covered elsewhere and they are not what this page is for.

Every definition is written to stand on its own, so it makes sense quoted somewhere else without the page around it. Where a term deserves more than a paragraph, the entry links to a full article or to a step-by-step guide in our knowledge base.

The definitions are maintained by the QA Madness team, who build QAM Hub and run test management on client projects for a living. Each entry carries a review date and we revisit the set on a schedule rather than when someone notices a mistake.

Filter terms

Automation coverage

Automation coverage is the share of documented test cases that are executed by automated tests rather than by people. It is usually reported per project, suite or release, sometimes weighted by priority. The figure is only trustworthy when automated results are mapped to test case records, so a case counts as automated because a result was recorded against it.

Full explanation of automation coverage

Custom field

An extra data field added to a test case, run or result to hold information the default schema does not cover: component, risk level, regulatory reference, automation owner. Custom fields are how a generic tool becomes yours. They are also how reporting gets useful, because a field you cannot filter on is a field you cannot report on.

Custom status

A team-defined execution outcome added to the built-in set of passed, failed, blocked and skipped, such as Not applicable, Deferred or Known issue. Custom statuses exist because forcing a real outcome into the wrong bucket corrupts every pass-rate figure downstream. The decision that matters is how each one counts in reports: as a pass, as a fail, or as neither.

Also called: Custom execution status, Custom result status

Full explanation of custom status

Defect linking

The connection between a test result and the bug record it produced in the issue tracker. Two-way linking means the tester sees the bug from the result and the developer sees the failing case from the bug. Without it, the question of which tests fail because of a given open bug has no answer short of asking people.

Also called: Bug linking, Issue linking

Full explanation of defect linking

Duplicate test case

A test case that checks behavior another case in the suite already checks, often written because searching the library took longer than writing a new one. Duplicates inflate coverage figures, double the maintenance on every requirement change and make one defect look like several failures. They rarely announce themselves, so finding them takes a deliberate review pass.

Also called: Redundant test case

Full explanation of duplicate test case

Execution status

The recorded outcome of running one test case in one test run: passed, failed, blocked, skipped, retest, or whatever custom values a team defines. The status belongs to the run, not to the case, which is why the same case can be passed in one release and failed in the next without either record overwriting the other.

Also called: test status, result status

Flaky test

A test that passes and fails against the same code, usually because of timing, shared state, test data or an unstable environment. Flakiness costs twice: every false failure takes someone's time to investigate, and the pattern teaches the team to ignore red builds. Detecting it needs execution history across many runs, which is why a single CI report cannot show it.

Also called: Intermittent test, Non-deterministic test

Full explanation of flaky test

JUnit XML report

A JUnit XML report is an XML file of test suites and test cases with each case's name, duration and outcome, plus failure messages and stack traces. It started with Java's JUnit and Ant and became the de facto exchange format for test results, so most runners can write it and most CI servers and test management tools can read it.

Also called: JUnit XML, XML test report

Full explanation of junit xml report

MCP server

An MCP server is a program that exposes a system's data and actions to AI assistants through the Model Context Protocol. For test management, it lets an assistant such as Claude or Cursor search test cases, create runs or record results directly, acting with the permissions of the token it was configured with.

Also called: Model Context Protocol server

Full explanation of mcp server

Milestone

A named target in a test management system, usually a release or a sprint, that groups the test runs belonging to it under a due date. The milestone is where progress against a deadline becomes visible: how much planned testing is done, what failed, and whether the remaining work fits the time left. Without one, release status gets rebuilt by hand from run names.

Also called: Release milestone

Full explanation of milestone

Precondition

The state the system and the data must be in before a test case's first step makes sense: a logged-in user, a seeded order, a specific feature flag. Preconditions written down are the difference between a case anyone can execute and a case only its author can. They are also the first thing to check when a case fails for everyone new.

Also called: setup, given

Quarantine

Test quarantine is the practice of moving an unreliable automated test out of the gating path while it keeps running and recording results. The build no longer fails because of it, but the team still sees how it behaves. A quarantine only works with an owner, a review date and a clear way out: fixed, rewritten or deleted.

Also called: Test quarantine, Muted test

Full explanation of quarantine

Regression suite

The set of test cases run to confirm that existing behaviour still works after a change. It grows by default and shrinks only on purpose, which is why mature teams review it on a schedule and retire cases that no longer earn their execution time. A regression suite nobody prunes eventually becomes too slow to run when it matters.

Also called: regression pack

Requirement

A statement of what the product must do, held in a tracker, a specification or the test management system itself. In testing terms a requirement matters because it is the thing coverage is measured against. A requirement with no linked test cases is untested by definition, and finding those gaps is most of what traceability is for.

Also called: user story, acceptance item

Requirements traceability matrix (RTM)

A view linking every requirement to the test cases that verify it and the results of those cases, so gaps and orphans are visible at a glance. Built by hand in a spreadsheet it is accurate on the day it is made and wrong within a sprint. Generated from live links, it stays accurate without anyone maintaining it.

Also called: traceability matrix, RTM

Sanity testing

A narrow check that one specific fix or change behaves as expected, usually run after a build is already known to be broadly stable. Sanity testing is deep and narrow where smoke testing is shallow and wide. The two get used interchangeably in conversation, which is harmless until someone reports "sanity passed" and the other person hears full coverage.

Smoke testing

A short, broad run confirming a build is stable enough to be worth testing properly: the application starts, the main paths work, nothing is obviously on fire. Smoke suites earn their place by being fast and by being run on every build. A smoke suite that takes forty minutes has stopped being a smoke suite.

Also called: build verification test

Tag

A free-form label attached to test cases so they can be selected across the suite hierarchy: smoke, mobile, payments, needs-data, slow. Tags are how you build a run that folders cannot express. They rot faster than any other organising device, so agree a small vocabulary early and audit it, or you end up with three spellings of one idea.

Also called: label, marker

Test case

The documented check of one behaviour: a title, preconditions, steps with expected results, and whatever metadata the team needs to filter on. A test case is a reusable asset executed many times against many builds, which is the property that separates it from a task or a ticket, and the reason issue trackers model it badly.

Also called: test script, test scenario

Test case ID

A test case ID is the permanent identifier a test management system gives each documented test case. It stays the same when the case title, steps or suite change, which makes it the reliable key for linking automated tests, defects and requirements to the case. Automation usually carries the ID in the test title, a tag or an annotation.

Full explanation of test case id

Test coverage

How much of something is exercised by tests, where the something has to be named to mean anything: requirements, features, code, risk. Requirement coverage is what test management systems measure, and it answers whether anything shipped untested. It is not code coverage, and quoting one number while meaning the other is the most common reporting error in QA.

Also called: requirement coverage

Test data

The inputs and system state a test needs to run: accounts, records, files, fixtures. Test data is the most common cause of a case that only its author can execute and of a run that fails in one environment and passes in another. Managing it means deciding what is seeded, what is generated and what is shared.

Also called: fixtures, seed data

Test management system (TMS)

The system of record for testing: where test cases live, where runs are executed and recorded, where automated results are collected, and where coverage and release readiness are reported. It sits between the issue tracker that holds requirements and defects, and the frameworks and pipelines that execute automated tests.

Also called: test case management tool, TCMS

Test plan

The document covering what will be tested for a specific release or project, by whom, in which environments, on what schedule, and what is explicitly out of scope. A plan is bounded and dated. It answers "what are we doing this time", which is a different question from the one a test strategy answers.

Test result

The record of one execution of one test case in one run: its status, who or what ran it, when, how long it took, and the evidence attached. Results accumulate into history, and history is what makes flakiness, stability trends and pass-rate comparisons possible. A tool that stores only the latest status has thrown that away.

Test run

A selected set of test cases executed together for a purpose, against a specific build and environment, with an owner and a result per case. The run is the unit that carries execution history. Creating a run rather than editing statuses on the cases themselves is what lets you compare this release with the last one.

Also called: test cycle, test execution

Test step

One action within a test case, paired with the result expected after it. Steps make a case executable by someone who did not write it, and they make a failure precise: the run records which step failed, not just that the case did. Cases with a single vague step are the usual reason failures need re-investigation.

Also called: step, test action

Test strategy

The standing approach to testing across an organisation or product: what gets automated, which risks justify which depth, which environments are maintained, how defects are triaged. A strategy outlives releases. A test plan applies it to one of them, which is why copying strategy text into every plan is a sign that neither document is doing its job.

Test suite

A named grouping of test cases, usually by feature, component or type of testing, forming the structure a team browses and selects from. Suites organise the library; runs execute from it. Keeping that distinction is what stops a suite from becoming a record of one release's execution and losing its reusability.

Also called: test folder, test collection

Traceability

The maintained links between requirements, test cases, runs, results and defects, so any one of them can be followed to the others. Traceability turns a question like "was this requirement tested, and did it pass in the release we shipped" into a lookup rather than an investigation. Regulated industries require it; everyone else needs it at audit time.

Version history

The record of how a test case changed over time, who changed it, and the ability to view or restore an earlier version. It matters when a case starts failing and nobody remembers whether the product changed or the test did. Without it, a test case is a document with no accountability, and disputes about it are settled by recollection.

Full explanation of version history