Flaky test detection & quarantineNew
Every automated test gets a pass rate, flaky rate and stability score across runs, and a known-flaky test can be quarantined until it is fixed.
A flaky test is worse than a failing one: it teaches the team to ignore red. QAM Hub keeps a record of every automated test across every uploaded run, so instability shows up as a number instead of a feeling, and a test you already know about can be set aside until it is fixed.
What counts as flaky
Each automated test is registered once, by its full path and its browser or configuration, and every run adds a result to that record. A result is flaky when the framework reported it as flaky or when the test passed only after a retry. JUnit XML reruns are folded into one test with its attempts, so a retried test reads as flaky rather than as a failure followed by a pass.
The numbers on the Tests tab
| Metric | What it counts |
|---|---|
| Pass rate | Runs that ended green, including passes after a retry |
| Flaky rate | Runs that needed a retry or were reported as flaky |
| Fail rate | Runs that ended red |
| Stability score | Clean passes on the first attempt only |
Skipped runs are left out of every rate, so a test that was switched off for a week does not look unstable. Next to each test sit a sparkline of its recent outcomes and its average duration; open one for its run-by-run history, duration trend and top failure reasons.
Choose what to measure
Measure over the last 5 to 100 runs, a period such as this week or this month, a date range, or a hand-picked set of runs. Narrow by framework, linked suite, browser or configuration, tests with no manual case, or free text. The flaky count on a run card opens the same view scoped to exactly the tests of that run.
Quarantine
A Project Admin can quarantine a test the team already knows is unstable, with a reason everyone can read, from the Tests tab, from the test's history or from the test details of any run. It stays visible and flagged with who quarantined it and when, and it is left out of the Tests tab averages, the flaky count and the failure analysis in Reports, so the same known problem stops coming back as news while someone fixes it.
Quarantine does not rewrite history or touch CI. A run keeps the pass rate your pipeline reported, with its quarantined failures marked as known, and the test keeps running until you change it in code. A filter lists every quarantined test, including ones that no longer run, so nothing stays set aside by accident.
Works with your stack
Playwright and Cypress report through the official npm reporters. WebdriverIO, Selenium, Robot Framework, pytest, JUnit, TestNG, NUnit and any other framework that writes JUnit XML report through the qam-hub CLI.