Skip to content

Software Quality

Glossary

The vocabulary the method pages rely on: terms the project coins or narrows, and the industry shorthand those pages assume.

For mappings to the wider testing vocabulary — unit / integration / e2e, mock vs stub vs fake, approvalsnapshotgolden master, classicist vs mockist — see Conventional terminology.

Project terms

  • Oracle. Anything that decides whether an output is correct. In an example test, the oracle is the author's expected output hard-coded in the assertion. In property-based testing, the oracle is a property. In differential testing, the oracle is another implementation. In metamorphic testing, the oracle is a relation. In fuzzing with a sanitizer, the oracle is the sanitizer.
  • Effect. What the code under test touches — the code axis; it determines which verification method fits. Inspired by Koka's effect rows, but regrouped for testing, with nondeterminism split into magnitude tiers.
  • Effect scope. A method's effect ceiling: the richest code-effect (state, I/O, nondeterminism tier) it can verify faithfully without quietly reducing the system. The Effect axis read method-side: for code with a given effect, the adequate methods are those whose scope covers it. A separate question from how much of its space a run exercises (reachability) and how strong a claim a pass makes (guarantee).
  • Effectiveness. A test suite's ability to detect faults — the field's term for "how good are the tests." It decomposes through the RIPR chain and behaves like a product, so a coverage percentage (one factor) is a poor proxy. See measuring test-suite effectiveness.
  • RIPR. Why a test catches a fault, in four steps; coverage reads only the first, the oracle is the last, and mutation testing exercises the whole chain (see measuring test-suite effectiveness):
    • Reachability: the test executes the faulty code.
    • Infection: executing it produces a wrong program state.
    • Propagation: the wrong state reaches an observable output.
    • Revealability: the oracle checks that output.
  • Adequacy. Criterion-relative thoroughness: a suite is "X-adequate" if it satisfies criterion X (statement, branch, MC/DC, mutation). The adequacy ladder orders criteria by demandingness and refines the guarantee axis.
  • Reachability. Whether a run executes the code, path, or state in question: what coverage measures, and the first factor of RIPR. Also the refinement beneath the guarantee tier: how much of a method's own space a run exercises. Distinct from a method's effect scope (which effects it covers, not how much of them).
  • Guarantee. The strength of claim a passing run licenses (also said it warrants, underwrites, or vouches for the claim): a per-method tier ordered by soundness, from a guess to a proof. Capped by the input and oracle axes and refined by adequacy. See guarantee.
  • Predicate. A boolean function of a value (x>0x > 0) — the position-neutral atom beneath several named roles. As a shape constraint it is a type (x:Tx : T); a precondition, postcondition, invariant, property, constraint, or refinement is the same predicate placed at a position or discharged a particular way. See contracts as specifications.
  • Invariant. Overloaded across communities; each sense means holds constant as something varies, and the same word names different cuts (the oracle-axis sense f(x)0f(x) \geq 0 is, in contract terms, a postcondition, not the lifetime invariant; see contracts as specifications):
    • Across inputs: on the oracle axis and in property-based testing, a predicate that holds for every input, x. P(f(x))\forall x.\ P(f(x)) (sort(xs) is ordered).
    • Across a lifetime: in Design by Contract, a predicate on an object's state, holding before and after every public method.
    • Across iterations: in Hoare logic, a loop invariant.
  • Regression ceiling. The structural limit on what example-based tests can catch: example tests guarantee the absence of known bugs, not the absence of bugs — see Example tests.
  • Ceiling-breaking method. Any method whose oracle isn't the test author's intuition: property-based testing, fuzzing, differential testing, metamorphic testing, formal methods, types, contracts. Defined opposite to example-based; framing in Example tests.
  • Hygiene tool. A tool that improves consistency without reasoning about runtime behavior: formatters, stylistic lint rules, naming conventions. Useful but not verification.
  • Verification method. A tool or practice that reasons about runtime behavior. Types, contracts, tests of all kinds, formal methods, code review, runtime monitoring, fuzzing.
  • Soundness and completeness (analysis). Sound: no false negatives within scope, so the analysis never reports "ok" when the bug it targets is present. Complete: no false positives, never flagging a non-bug (the dual). A tool is usually one, not both. This is the program-analysis property behind the guarantee tiers. See abstract interpretation and termination analysis.
  • Sound type system vs unsound type system. A type system is sound if a passing check guarantees the absence of the type errors the system claims to forbid. Haskell and OCaml are sound (within their expressive reach). TypeScript and mypy are unsound by design — gradual typing keeps backward compatibility with dynamic code. See Static types.
  • By construction. Of a guarantee: holds because of the structure of the code or the verifier, not because we observed it on test inputs. Memory safety in Rust is by construction; memory safety in C with AddressSanitizer is empirical.
  • Folklore. The project's evidence tier for claims that are widely cited but unsupported by primary studies. Parallel categories:
    • Testing folklore: the pyramid as prescription, the 80% coverage target, TDD as universal design discipline, the 5–8× PBT productivity multiplier, "100% coverage subsumes types".
    • Design folklore: small files as a universal prescription, cyclomatic complexity as a quality KPI, GoF patterns as a universal design vocabulary, DRY as no duplication.

Industry shorthand

Terms with a settled meaning across the engineering literature. The method pages use them without further definition.

  • Shift-left. Move quality work earlier in the development cycle, where defects are cheaper to fix. The framing dates to Fagan-era inspection literature and pre-dates "shift-left" as a marketing term.
  • TDD — test-driven development. Red-green-refactor cycle. The project treats TDD's empirical record as mixed; the "TDD as universal design discipline" section in Testing folklore carries the case-by-case analysis.

Referenced by