# The test suite as an object

A test suite is an artifact with its own qualities, and "a good suite" means
several different things at once, each with its own measure and its own remedy.
Most methods in this catalog reduce uncertainty about the *code*; the methods
and measures here turn the question on the suite itself.

## How effective is it?

Whether the suite actually catches faults is its **effectiveness**, and
[measuring test-suite effectiveness](https://quality.stereobooster.com/measuring-test-effectiveness.md) sets out the
RIPR chain behind it, why [coverage](https://quality.stereobooster.com/coverage.md) measures only
reachability (a weak proxy), and where the real-fault benchmarks sit. The sharper
single measure is [mutation testing](https://quality.stereobooster.com/mutation-testing.md), which exercises the
whole chain rather than just reach.

## What does it cost to run?

Runtime is not fixed. When a suite grows slow enough to throttle feedback,
[regression-suite management](https://quality.stereobooster.com/regression-management.md) buys the time back three
ways: [prioritization](https://quality.stereobooster.com/test-case-prioritization.md) reorders so likely failures
surface first, [selection](https://quality.stereobooster.com/regression-test-selection.md) skips the tests a change
cannot affect, and [minimization](https://quality.stereobooster.com/test-suite-minimization.md) deletes redundant
tests outright (the most aggressive, and the one to treat with care).

## Can you trust its verdict?

An unstable verdict corrodes every other measure. [Flaky tests](https://quality.stereobooster.com/flaky-tests.md)
covers what causes them, how to provoke them deliberately rather than wait for
them, and what to do with one. The suite-level number is the *flake rate*: the
share of runs that fail on a retry of the same commit, which the
[maintainability dimension](https://quality.stereobooster.com/maintainability.md) tracks
against a target.

## Which tests need attention?

The per-test history that steers prioritization also ranks the
suite's own problems: the tests that fail flakily, fail most often, or run slowest
are the ones worth deflaking, splitting, or deleting first. This is the test-suite
counterpart of [git-history hotspots](https://quality.stereobooster.com/git-hotspots.md) for production
code, and a test that never fails and covers nothing new is a
[minimization](https://quality.stereobooster.com/test-suite-minimization.md) candidate.

## Is it maintainable?

Beyond speed and stability, a suite can be hard to read or brittle to change, and
a failure can point cleanly at its cause or not. Diagnosability and the rest are
the [other
qualities](https://quality.stereobooster.com/measuring-test-effectiveness.md#other-dimensions-of-test-quality) on
which a suite is judged; the smell catalog that names test-design problems, and
how far its evidence reaches, is covered under [test
smells](https://quality.stereobooster.com/test-smells.md).

## Referenced by

- [Methods](https://quality.stereobooster.com/methods.md) · Methods

## Acronyms

- RIPR — reachability, infection, propagation, revealability