A test suite is an artifact with its own qualities, and "a good suite" means several different things at once, each with its own measure and its own remedy. Most methods in this catalog reduce uncertainty about the code; the methods and measures here turn the question on the suite itself.

How effective is it?

Whether the suite actually catches faults is its effectiveness, and measuring test-suite effectiveness sets out the RIPR chain behind it, why coverage measures only reachability (a weak proxy), and where the real-fault benchmarks sit. The sharper single measure is mutation testing, which exercises the whole chain rather than just reach.

What does it cost to run?

Runtime is not fixed. When a suite grows slow enough to throttle feedback, regression-suite management buys the time back three ways: prioritization reorders so likely failures surface first, selection skips the tests a change cannot affect, and minimization deletes redundant tests outright (the most aggressive, and the one to treat with care).

Can you trust its verdict?

An unstable verdict corrodes every other measure. Flaky tests covers what causes them, how to provoke them deliberately rather than wait for them, and what to do with one. The suite-level number is the flake rate: the share of runs that fail on a retry of the same commit, which the maintainability dimension tracks against a target.

Which tests need attention?

The per-test history that steers prioritization also ranks the suite's own problems: the tests that fail flakily, fail most often, or run slowest are the ones worth deflaking, splitting, or deleting first. This is the test-suite counterpart of git-history hotspots for production code, and a test that never fails and covers nothing new is a minimization candidate.

Is it maintainable?

Beyond speed and stability, a suite can be hard to read or brittle to change, and a failure can point cleanly at its cause or not. Diagnosability and the rest are the other qualities on which a suite is judged; the smell catalog that names test-design problems, and how far its evidence reaches, is covered under test smells.

Referenced by

• Methods · Methods