Skip to content

Software Quality

Functionality

Does the software do what it claims?

Behavior matching what the software claims to do. The most-discussed dimension in software quality and the one the testing tradition lives inside. Two classes of method divide the work: example tests pin the cases written down, and generative methods (property-based, fuzzing, differential, metamorphic) reach the input space enumeration can't cover. A functionality verification program carries both.

What goes wrong

  • Incorrect output for known inputs. Example regression tests catch these, once written.
  • Incorrect output for inputs nobody thought to test. Where example enumeration falls short by construction. Property-based, fuzzing, differential, and metamorphic testing target this gap.
  • Edge cases. Off-by-one, null/empty/overflow, time zones, the boundary of integer ranges. Often invisible to spec writers because the spec doesn't enumerate them.
  • Concurrency and ordering. Race conditions, lost updates, out-of-order delivery. TLA+ and deterministic simulation find these; example tests miss them.
  • API contract drift. Two services that agreed to a spec but one drifted. Static schema-compatibility checks (OpenAPI diffing, protobuf breaking-change detection) catch syntactic breaks; live integration tests catch semantic ones; consumer-driven contract tests catch semantic breaks without a live counterpart (snapshot testing at the seam).

What success looks like

  • Bug-fix commits include a test. The regression ceiling means this is the only way to know the bug won't recur.
  • Diff-time signals fire before merge. Coverage, linter, and types on touched files.
  • Property tests cover the input space that can be described. A bug that falls outside that space marks the next gap.
  • The next bug class has a named oracle. For input classes that can't be enumerated, a property, a reference implementation, or a metamorphic relation is the oracle to reach for next.

Referenced by