Prioritization orders the whole suite so that the tests most likely to reveal a fault run first. It removes nothing and skips nothing, so it carries no correctness risk: run long enough and every test still executes. Its only lever is when you find out, which makes it the lowest-risk of the regression-suite management techniques.

What it does

On a suite that takes an hour, an ordering that surfaces the first failure two minutes in rather than fifty gives the developer the same information forty-eight minutes earlier, with the total run time unchanged. The benefit is measured by APFD, the rate at which the ordering accumulates fault detection.

Coverage- and history-based ordering

• Coverage-based. Order tests by the coverage each one contributes. Total greedy sorts tests by the coverage each achieves on its own; additional greedy repeatedly picks the test that adds the most not-yet-covered code, recomputing after each pick. Coverage-based ordering needs instrumentation and a stored coverage profile per test, so it costs something to maintain.
• History-based. Order by past behavior: tests that failed recently, that fail often, or that have not run in a while go first. It needs no instrumentation, only the CI system's own pass/fail log. That is what makes it applicable under continuous integration, where coverage-based techniques are hard to apply: they assume instrumentation and a discrete, complete test suite (Elbaum et al. 2014)¹.

Tools

No dedicated tool is required: a history-based ordering is a few lines of scheduling code over the CI system's own pass/fail log, ordering by last-failure time or failure frequency. What build tools and hosted CI ship instead is dependency-based selection, which skips tests rather than reordering them.

When to use, when not

Use it almost always. An ordering that turns out badly costs only the time the run would have taken anyway.

Don't reach for coverage-based ordering when history-based will do. The extra instrumentation rarely pays for itself in a CI setting.

Evidence

• Prioritization improves the fault-detection rate. In the family of empirical studies that established the area, every technique considered improved that rate overall (Elbaum et al. 2002)². Feedback alone does not decide the ranking: additional statement coverage came last among the statement-level heuristics compared, significantly worse than total coverage, while the technique that ranked first combined additional coverage with an estimate of each test's fault-exposing potential. The gains are real but bounded, and the ranking of heuristics is context-dependent rather than universal.
• Ordering by failure history shortens the time to a failure report, measured against no prioritization on a Google dataset of millions of test-suite executions (Elbaum et al. 2014)¹.

Classification

• Quality dimensions: Maintainability (shortens feedback latency by running likely-failing tests first; no correctness risk) — orders an existing suite so likely failures surface first; it schedules tests, it does not run or verify them, so it sits off the code-verification axes.
• Area: Long regression suites where the wait for the first failure matters: continuous-integration pipelines and end-to-end suites.

Referenced by

• Regression-suite management · Methods
• The test suite as an object · Methods

References

---

1. Elbaum, Sebastian, Gregg Rothermel, and John Penix. 2014. "Techniques for Improving Regression Testing in Continuous Integration Development Environments." Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2014), 235–45. https://doi.org/10.1145/2635868.2635910. ↩↩

2. Elbaum, Sebastian, Alexey G. Malishevsky, and Gregg Rothermel. 2002. "Test Case Prioritization: A Family of Empirical Studies." IEEE Transactions on Software Engineering 28 (2): 159–82. https://doi.org/10.1109/32.988497. ↩