Quarantine
Test quarantine is the practice of moving an unreliable automated test out of the gating path while it keeps running and recording results. The build no longer fails because of it, but the team still sees how it behaves. A quarantine only works with an owner, a review date and a clear way out: fixed, rewritten or deleted.
Also called: Test quarantine, Muted test
Quarantine is not the same as skipping
A skipped test produces no data. A quarantined test keeps executing and keeps recording results; it just stops deciding whether the build passes. That difference is the whole point. Without results you cannot tell whether a fix worked, whether the test is getting worse, or whether it has started failing for a real reason. Teams usually implement this as two CI signals from one suite: a gating job that excludes quarantined tests and a non-blocking job that runs only them, without retries.
The rules that keep it honest
A quarantine without rules fills up and stays full. Four rules do most of the work:
- every quarantined test has one named owner and a review date;
- the record says which failure it showed and which test case or requirement it was covering;
- the quarantine has a size cap, so adding a test forces a decision about another;
- a test leaves by being fixed, rewritten at a lower level, or deleted with its lost coverage recorded.
What teams underestimate
The coverage loss. A quarantined end-to-end test may be the only automated check behind a regression case, and nothing in a CI log says so. Mapping automated results to test cases is what makes that gap visible. The mechanics are in mapping automated test results to test cases, and the full policy with starting values is in a flaky test quarantine policy that actually holds.