The testing literature has accreted decades of vocabulary — unit test, integration test, regression, acceptance, smoke, mock, stub, golden master. The terms are useful as shorthand but cluster meanings that often differ between authors.

Conventional taxonomy → the axes

The unit/integration/e2e hierarchy is the dominant mental model in software testing. The project's pages use its terms — but the labels package multiple axes into one, which is the source of most "what counts as a unit test?" debates.

Term	Input	Oracle	Guarantee	Ambiguity
Unit test	Fixed	Any	Empirical	Packs narrow in scope with isolated by mocks. The two aren't the same; most "unit tests" should arguably be behavioral tests against a port (Cooper 2013)1.
Integration test	Fixed	Any	Empirical	"Integration of what?" In some shops it means "two modules"; in others "module + DB"; in others "module + a real third-party service."
End-to-end (E2E) test	Fixed	Any	Empirical	Often used interchangeably with system test, acceptance test. Strictly: drives the whole deployed system from its outermost surface.

All three rows carry the same defaults (a fixed input, an oracle the author writes as code, an empirical guarantee) unless one is chosen otherwise. The levels differ only in scope (how much code a run exercises) and size (the resources and I/O reach it needs), properties of the code and the run that Google's taxonomy names directly and the method axes deliberately leave out.

Scope and size (Google's taxonomy)

Google splits the two properties the unit/integration/e2e labels conflate, and treats them as orthogonal (Winters et al. 2020)²:

• Scope — how much code a run exercises: narrow (one class or function), medium (a few components together), broad (the whole system).
• Size — the resources a run is allowed, which is what actually governs its speed and flakiness: small (one process, one thread, no I/O, no sleeps — fast and deterministic), medium (one machine: localhost, a local database, the filesystem), large (multiple machines, real network, external services).

Conventional term	Scope	Typical size
Unit	narrow	small
Integration	medium	medium
End-to-end	broad	large

The size column is only a default. The two properties come apart: a test that touches the disk or the clock is medium in size however narrow its scope, and it is the size that makes it slow and flaky. Google therefore classifies by size first and targets mostly-small suites, whereas the pyramid assigns one label per level and so covers scope and size together.

Visibility: black-box, gray-box, white-box

Visibility is what a method can observe of the system while it runs: the bandwidth of the channel between the method and the running code. It is set by the instrumentation available, not by what the person driving the method knows about the implementation.

The three labels mark points on that one gradient rather than three kinds of testing, and each widening of the channel makes the next family of methods possible:

Channel	The method observes	What it unlocks
None	the output, and whether the process died	random generation, approval tests, synthetic probing
Crash and exit status	that something failed, not where	crash triage, black-box fuzzing, Jepsen-style fault testing
Coverage bits	which edges a run reached	coverage-guided (gray-box) fuzzing, coverage measurement, mutation testing
Path constraints	the symbolic condition under which a run took its path	concolic execution, branch-distance guidance in search-based testing
Source and semantics	the program text and its meaning	symbolic execution, abstract interpretation, types, proof

Visibility is a third property the pyramid labels conflate, independent of both scope and size. A unit test driving a public API with no instrumentation is black-box at narrow scope; an end-to-end run under a coverage agent, with a fault injected into one chosen internal call, is white-box at broad scope. The equation end-to-end = black-box, unit = white-box packs together things that come apart in practice.

It is not one of the axes, because it is a property of the situation rather than of the method: the same method sits at the same coordinates whether the source is available or not. Visibility decides which coordinates are available. A generative-feedback input needs a feedback channel, so against a genuine black box the input axis collapses to fixed or generative-random. Every criterion on the adequacy ladder needs instrumentation, so a coverage number is undefined without one. And a proof needs the semantics, which is why temporal-logic falsification exists at all: the Simulink model is opaque, so what remains is search against a robustness score.

More channel is not automatically more bugs found, and the fuzzing literature is weak ground for the comparison. A survey of 32 fuzzing papers found problems in the experimental evaluation of every one, and the authors' own experiments showed those problems can produce wrong or misleading assessments (Klees et al. 2018)³. What the channel buys is definitional rather than measured: a black-box fuzzer cannot run the feedback loop at all, because the signal it would steer by does not reach it.

Synonyms and near-synonyms

Words that look like distinctions but usually aren't, or that collapse together under different authors.

• Example test ↔ automanual test (Wayne 2018)⁴. The project uses example-based to emphasize the input axis; Wayne's automanual captures the same thing.
• Regression test = example test added after a bug fix. The distinguishing feature is the role in the suite, not the form. Every regression test is an example test; not every example test is a regression test.
• Approval test ↔ snapshot test ↔ golden master test. Same technique under three names. Golden master carries loaded vocabulary; the project standardizes on approval (or snapshot where the framework uses that term).
• Characterization test (Feathers 2004)⁵ = an approval test applied to legacy code with no specification. The test pins down current behavior to make a refactor safe; correctness is a separate question.
• BDD / Gherkin / Specification by Example — these are styles for writing example tests, not separate test families. Given/When/Then is a syntax; the test is still an example test underneath.
• Mock ↔ stub ↔ fake ↔ spy (Fowler 2007)⁶. Mock is routinely used for every kind of test double; Fowler separates them:
  • Stub — returns canned answers; tests against state.
  • Mock — verifies interactions; tests against calls.
  • Fake — a working implementation, just not the production one (in-memory DB, fake clock).
  • Spy — records calls for later assertion.
• Classicist vs mockist TDD — Fowler's terms for the two schools. Classicist: test against state with real collaborators or fakes. Mockist: test against interactions with mocked collaborators.

Referenced by

• The axes · The axes
• Conventional · Conventional
• Glossary · Overview

References

---

1. Cooper, Ian. 2013. TDD: Where Did It All Go Wrong? https://www.youtube.com/watch?v=EZ05e7EMOLM. ↩

2. Winters, Titus, Tom Manshreck, and Hyrum Wright, eds. 2020. Software Engineering at Google: Lessons Learned from Programming Over Time. O'Reilly Media. https://abseil.io/resources/swe-book/html/ch11.html. ↩

3. Klees, George, Andrew Ruef, Benji Cooper, Shiyi Wei, and Michael Hicks. 2018. "Evaluating Fuzz Testing." Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (CCS '18), 2123–38. https://doi.org/10.1145/3243734.3243804. ↩

4. Wayne, Hillel. 2018. Just a Whole Bunch of Different Tests. https://www.hillelwayne.com/post/a-bunch-of-tests/. ↩

5. Feathers, Michael. 2004. Working Effectively with Legacy Code. Prentice Hall PTR. ↩

6. Fowler, Martin. 2007. Mocks Aren't Stubs. https://martinfowler.com/articles/mocksArentStubs.html. ↩