The vocabulary the method pages rely on: terms the project coins or narrows, and the industry shorthand those pages assume.
For mappings to the wider testing vocabulary — unit / integration / e2e, mock vs stub vs fake, approval ↔ snapshot ↔ golden master, classicist vs mockist — see Conventional terminology.
Project terms¶
- Oracle. Anything that decides whether an output is correct. In an example test, the oracle is the author's expected output hard-coded in the assertion. In property-based testing, the oracle is a property. In differential testing, the oracle is another implementation. In metamorphic testing, the oracle is a relation. In fuzzing with a sanitizer, the oracle is the sanitizer.
- Effect. What the code under test touches — the code axis; it determines which verification method fits. Inspired by Koka's effect rows, but regrouped for testing, with nondeterminism split into magnitude tiers.
- Effect scope. A method's effect ceiling: the richest code-effect (state, I/O, nondeterminism tier) it can verify faithfully without quietly reducing the system. The Effect axis read method-side: for code with a given effect, the adequate methods are those whose scope covers it. A separate question from how much of its space a run exercises (reachability) and how strong a claim a pass makes (guarantee).
- Effectiveness. A test suite's ability to detect faults — the field's term for "how good are the tests." It decomposes through the RIPR chain and behaves like a product, so a coverage percentage (one factor) is a poor proxy. See measuring test-suite effectiveness.
- RIPR. Why a test catches a fault, in four steps; coverage reads only the
first, the oracle is the last, and mutation testing exercises the whole chain
(see measuring test-suite
effectiveness):
- Reachability: the test executes the faulty code.
- Infection: executing it produces a wrong program state.
- Propagation: the wrong state reaches an observable output.
- Revealability: the oracle checks that output.
- Adequacy. Criterion-relative thoroughness: a suite is "X-adequate" if it satisfies criterion X (statement, branch, MC/DC, mutation). The adequacy ladder orders criteria by demandingness and refines the guarantee axis.
- Reachability. Whether a run executes the code, path, or state in question: what coverage measures, and the first factor of RIPR. Also the refinement beneath the guarantee tier: how much of a method's own space a run exercises. Distinct from a method's effect scope (which effects it covers, not how much of them).
- Guarantee. The strength of claim a passing run licenses (also said it warrants, underwrites, or vouches for the claim): a per-method tier ordered by soundness, from a guess to a proof. Capped by the input and oracle axes and refined by adequacy. See guarantee.
- Predicate. A boolean function of a value () — the position-neutral atom beneath several named roles. As a shape constraint it is a type (); a precondition, postcondition, invariant, property, constraint, or refinement is the same predicate placed at a position or discharged a particular way. See contracts as specifications.
- Invariant. Overloaded across communities; each sense means holds constant
as something varies, and the same word names different cuts (the oracle-axis
sense is, in contract terms, a postcondition, not the lifetime
invariant; see contracts as specifications):
- Across inputs: on the oracle axis and in
property-based testing, a predicate that holds for every input,
(
sort(xs)is ordered). - Across a lifetime: in Design by Contract, a predicate on an object's state, holding before and after every public method.
- Across iterations: in Hoare logic, a loop invariant.
- Across inputs: on the oracle axis and in
property-based testing, a predicate that holds for every input,
(
- Regression ceiling. The structural limit on what example-based tests can catch: example tests guarantee the absence of known bugs, not the absence of bugs — see Example tests.
- Ceiling-breaking method. Any method whose oracle isn't the test author's intuition: property-based testing, fuzzing, differential testing, metamorphic testing, formal methods, types, contracts. Defined opposite to example-based; framing in Example tests.
- Hygiene tool. A tool that improves consistency without reasoning about runtime behavior: formatters, stylistic lint rules, naming conventions. Useful but not verification.
- Verification method. A tool or practice that reasons about runtime behavior. Types, contracts, tests of all kinds, formal methods, code review, runtime monitoring, fuzzing.
- Soundness and completeness (analysis). Sound: no false negatives within scope, so the analysis never reports "ok" when the bug it targets is present. Complete: no false positives, never flagging a non-bug (the dual). A tool is usually one, not both. This is the program-analysis property behind the guarantee tiers. See abstract interpretation and termination analysis.
- Sound type system vs unsound type system. A type system is sound if a passing check guarantees the absence of the type errors the system claims to forbid. Haskell and OCaml are sound (within their expressive reach). TypeScript and mypy are unsound by design — gradual typing keeps backward compatibility with dynamic code. See Static types.
- By construction. Of a guarantee: holds because of the structure of the code or the verifier, not because we observed it on test inputs. Memory safety in Rust is by construction; memory safety in C with AddressSanitizer is empirical.
- Folklore. The project's evidence tier for claims that
are widely cited but unsupported by primary studies. Parallel
categories:
- Testing folklore: the pyramid as prescription, the 80% coverage target, TDD as universal design discipline, the 5–8× PBT productivity multiplier, "100% coverage subsumes types".
- Design folklore: small files as a universal prescription, cyclomatic complexity as a quality KPI, GoF patterns as a universal design vocabulary, DRY as no duplication.
Industry shorthand¶
Terms with a settled meaning across the engineering literature. The method pages use them without further definition.
- Shift-left. Move quality work earlier in the development cycle, where defects are cheaper to fix. The framing dates to Fagan-era inspection literature and pre-dates "shift-left" as a marketing term.
- TDD — test-driven development. Red-green-refactor cycle. The project treats TDD's empirical record as mixed; the "TDD as universal design discipline" section in Testing folklore carries the case-by-case analysis.
Referenced by¶
- Contracts as specifications · Methods
- Software Quality · Overview