# Fuzzing

Fuzzing is an automated testing method that runs a program on generated
inputs and reports a crash, undefined behavior or a sanitizer violation
as a failure. Nothing has to state the expected output for any given
input, and that is what lets the input count run into the millions.

## What it catches

Fuzzing is most effective on *memory-safety* and *crash* bugs. With
the right oracles attached (sanitizers, invariants), it extends to
logic bugs.

- **Memory-safety bugs.** Buffer overflows, use-after-free,
  double-free, out-of-bounds reads. AddressSanitizer makes these
  observable.
- **Undefined behavior in C/C++.** UndefinedBehaviorSanitizer
  flags signed overflow, null deref, invalid shifts.
- **Concurrency bugs.** ThreadSanitizer catches data races; thread-aware
  fuzzers (Krace, MUZZ) steer the search into them.
- **Logic bugs surfaced by invariants.** If the code asserts
  invariants, fuzzing exercises them across an enormous input
  space.
- **Parser bugs.** Crashes, infinite loops, exponential-time
  inputs (CVE pattern; ReDoS for regex).
- **Protocol bugs.** State-machine fuzzers (e.g., AFLNet,
  syzkaller) catch deep protocol violations.

Fuzzing does **not** catch silent wrong-output bugs unless you
attach an oracle. A fuzzer that finds no crashes has not shown the
program correct.

## Tools

### Continuous fuzzing infrastructure

- **[OSS-Fuzz](https://google.github.io/oss-fuzz/)** — Google's continuous fuzzing service for open
  source. Free. Required for security-sensitive open-source C/C++
  that handles untrusted input.

### Coverage-guided fuzzers

- **[libFuzzer](https://llvm.org/docs/LibFuzzer.html)** — in-process, fast; ships with LLVM. Best for C
  and C++ APIs.
- **[AFL++](https://github.com/AFLplusplus/AFLplusplus)** — process-based; the AFL successor; standalone
  binaries.
- **[Honggfuzz](https://github.com/google/honggfuzz)** — similar trade-offs; alternative to AFL.

### Per ecosystem

- **C / C++:** libFuzzer, AFL++ + [AddressSanitizer](https://clang.llvm.org/docs/AddressSanitizer.html) +
  [UndefinedBehaviorSanitizer](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html) + [ThreadSanitizer](https://clang.llvm.org/docs/ThreadSanitizer.html).
- **Rust:** [cargo-fuzz](https://github.com/rust-fuzz/cargo-fuzz) (libFuzzer wrapper), afl.rs. Rust's
  ownership eliminates many memory bugs; fuzzing still catches
  logic and panic paths.
- **Python:** [Atheris](https://github.com/google/atheris) (Google).
- **Java / JVM:** [Jazzer](https://github.com/CodeIntelligenceTesting/jazzer) (Code Intelligence; libFuzzer-based).
- **Go:** native fuzzing in `go test` since 1.18.
- **JavaScript / TypeScript:** [Jazzer.js](https://github.com/CodeIntelligenceTesting/jazzer.js).

### Specialized fuzzers

- **[syzkaller](https://github.com/google/syzkaller)** — Linux kernel syscall fuzzer.
- **[AFLNet](https://github.com/aflnet/aflnet)** — protocol-aware fuzzing.
- **[Csmith](https://github.com/csmith-project/csmith)** — generates random C programs; the canonical
  compiler-fuzzing example.
- **[SQLancer](https://github.com/sqlancer/sqlancer)** — SQL fuzzing with metamorphic oracles for
  SQLite, MySQL, and PostgreSQL (Rigger and Su 2020)[^rigger2020].
- **[PerfFuzz](https://github.com/carolemieux/perffuzz)** / **[SlowFuzz](https://github.com/nettrino/slowfuzz)** — performance
  fuzzers: evolve inputs to *maximize execution cost* rather than to trigger a
  crash, surfacing worst-case algorithmic blow-ups
  ([algorithmic complexity testing](https://quality.stereobooster.com/algorithmic-complexity.md)).

### Concurrency fuzzers

Ordinary coverage feedback tracks which *code* a run reaches, not which
*interleaving* — so a gray-box fuzzer can hammer multithreaded code without
ever varying the schedule. These add interleaving to the feedback loop:

- **Krace** — coverage-guided data-race fuzzer for kernel file systems; it adds
  an interleaving-coverage signal alongside branch coverage, so the search
  explores thread schedules and not just inputs (Xu et al. 2020)[^xu2020].
- **MUZZ** — thread-aware gray-box fuzzing for multithreaded programs;
  instruments to stress the thread-interleaving states ordinary coverage ignores
  (Chen et al. 2020)[^chen2020].

These chase the same interleaving bugs as [systematic concurrency
testing](https://quality.stereobooster.com/systematic-concurrency-testing.md), but by coverage-guided
sampling rather than systematic or PCT-bounded exploration.

### Curated input corpora

- **[Big List of Naughty Strings](https://github.com/minimaxir/big-list-of-naughty-strings)** — a community-curated list of
  strings that have historically broken parsers and validators
  (Unicode edge cases, RTL marks, format-string syntax, SQL
  injection patterns, very long strings).
  Useful as a starter corpus for string-handling fuzz harnesses
  and for property-test generators that need realistic edge cases.

## When to use, when not

**Use:**

- Any code that parses untrusted input (network protocols, file
  formats, deserialization, regex compilation).
- Security-sensitive C/C++ code regardless of input source.
- Open-source libraries that handle parsing or crypto — submit
  to OSS-Fuzz.
- Linux kernel paths or syscall interfaces — syzkaller.

**Don't:**

- As your only verification. Fuzzing finds crashes; logic bugs
  need oracles you supply (properties, differential references,
  invariants).
- For pure business logic with no crash risk. The marginal value
  is low compared to property tests.
- For UI code, where crashes are caught by the platform and the
  failure modes are visual.

## Running a productive campaign

- **Mutate at semantic boundaries, not raw bytes.** Blind bit-flips
  and truncation die at the first validity check and never reach
  interesting code. The bugs come from handing the target a
  *plausible-but-wrong structure* it will follow — corrupt a length
  or offset field, a record count, a type tag: values the parser
  trusts. This is the case for *structure-aware* fuzzing: encode the
  format so the fuzzer mutates fields, not bytes (Google n.d.)[^googlend]; see
  also FormatFuzzer (Dutra et al. 2021)[^dutra2021].
- **Make the contract strict — "typed error or clean," never just
  "didn't crash."** "The process stayed up" is too weak an oracle:
  it passes a parser that leaks a raw `RangeError` /
  `NullPointerException` to its caller, or one that hangs. Assert
  instead that every rejection is the library's *own* typed error
  and that nothing hangs. A raw runtime error or a timeout becomes a
  *finding*, not a pass — which is what turns a crash fuzzer into a
  robustness fuzzer without a full differential oracle.
- **Iterate by layer: fix a class → re-fuzz → next layer.** You
  cannot see bug N+1 until bug N stops throwing. A binary reader
  peels in layers — bounds error, then over-allocation, then a
  corrupt count driving a huge array, then a view over a short
  slice — each hidden behind the previous. Budget several rounds.
- **Seed diversely.** A handful of structurally *different* valid
  inputs reaches far more code than one input mutated repeatedly.
- **Profile the oracle, not the RNG.** When a campaign is slow the
  bottleneck is almost always per-iteration work — building the
  input or an expensive oracle — not input generation. An oracle that
  materializes a large result the property never inspects can dominate
  wall-clock; bounding the inputs so the oracle stays cheap can cut
  runtime by an order of magnitude at the same coverage. Profiling the
  phases comes before parallelizing them.
- **Keep a fast gate *and* a deep campaign; don't consolidate.**
  A [property-based](https://quality.stereobooster.com/property-based-testing.md)
  generator ([fast-check](https://fast-check.dev/), [Hypothesis](https://hypothesis.readthedocs.io/)) gives a structured oracle and
  *shrinks* a failure to a minimal counterexample — run it with a
  fixed seed as an always-on CI gate (seconds). A coverage-guided
  byte fuzzer (libFuzzer, Jazzer/Jazzer.js, AFL++) steers into deep
  branches blind mutation can't reach — run it as a non-deterministic
  campaign against a corpus and a time budget. Neither subsumes the
  other.
- **The harness is the asset, not the fix list.** A list of bugs
  fixed is a snapshot; the harness keeps finding the next layer and
  guards against regressions. Keep it in the repo and raise the run
  count on a schedule. Fuzz *both* doors where untrusted input
  crosses in: the read side (arbitrary bytes) and the write side
  (arbitrary input → builder, then a build→read round-trip property).

## Recurring bug classes in parsers and binary readers

- **Sequential reads off the end.** Bound the *shared* read
  primitive once, so an out-of-bounds read throws the typed error,
  not a raw `RangeError`. Wrapping each call site individually is
  what *creates* the gap; enforce the bound where bytes become
  values.
- **Allocation sized by an attacker-controlled length word.** A
  corrupt length field drives an enormous allocation — and an
  *in-range* multi-GB request succeeds before it OOMs, so an
  "is it allocatable at all" guard misses it. Clamp the read to the
  known end of input at the chokepoint rather than trusting the
  word. For speculative/prefetch reads, *clamp* (resolve truncated)
  rather than *throw*, so a fire-and-forget read can't reject
  unhandled; surface the corruption typed on the awaited path.
- **Sentinel-terminated decode loops.** A loop that scans "until the
  terminating bit/byte" runs forever past a truncated region (the
  out-of-range read returns a value that never matches the sentinel)
  → a single input hangs the process → denial of service. Bound
  every such loop by a *counted* length, never the sentinel alone.
  Grep byte-stream code for `for (;;)` / `while (true)`.
- **Lazy "ensure-then-read" windows.** An accessor that assumes a
  prior ensure step ran must still bounds-check its index against the
  real count — corruption reaches the read without the ensure. Every
  such accessor needs a typed corruption guard.
- **Cross-structure invariants are reachable by corruption.** When
  the set a read *assumes was prepared* comes from a different
  on-disk structure than its read key, corrupt bytes can desync the
  two, firing an internal "you must call X first" assertion (meant as
  a programming-bug signal) on *data* corruption. Give those guards
  the typed corruption error too, not a bare `Error`.
- **Every corrupt-input guard must throw the library's typed error.**
  A guard that throws a bare `Error` reads as a crash to the harness
  and to the caller. One typed error type at the deserialize boundary
  fixes most of these classes at once.

## Evidence

- **OSS-Fuzz** public counters (May 2025): >13,000 vulnerability
  fixes, >50,000 bug fixes across >1,000 projects (Google 2025)[^ossfuzz].
- **syzkaller.** One independent analysis of 125,000+ kernel
  vulnerabilities found that the share of Linux kernel bugs fixed
  within a year of being introduced rose from ~0%
  (2010) to ~69% (2022), attributing the shift to syzkaller
  plus KASAN/KMSAN/KCSAN (Qu 2026)[^qu2026]; the primary source is a blog
  analysis of kernel git history, not a peer-reviewed study —
  the direction of the effect is well-attested, the exact
  percentages should be treated as indicative.
- **Csmith** found 325+ unknown compiler bugs in GCC, LLVM, and
  others over three years (Yang et al. 2011)[^yang2011].
- **The [CompCert](https://compcert.org/) outlier**: Csmith found no wrong-code bugs in its
  formally [verified](https://quality.stereobooster.com/theorem-proving.md) middle-end (Leroy 2009; Yang et al. 2011)[^leroy2009] [^yang2011] —
  the contrast that bounds what fuzzing alone reaches.

## Related

**Exploring the program's behavior space**

All explore many program behaviors; they differ in what they run and what they
guarantee. [Model checking](https://quality.stereobooster.com/model-checking.md) works on a spec or
model and is exhaustive over it. [Symbolic
execution](https://quality.stereobooster.com/symbolic-execution.md) runs the *actual code*, solving
path constraints to reach branches a fuzzer can't, with a bounded-exhaustive
verdict; [bounded model checking](https://quality.stereobooster.com/bounded-model-checking.md) runs
the actual code too, but poses a single SMT query over the whole program unrolled
to a fixed depth rather than exploring paths one at a time.
Fuzzing runs the actual code too but samples inputs
empirically: cheaper per case, no exhaustiveness claim.

**Perturbation testing: what you perturb, and where**

All three deliberately subject a system to bad conditions, differing in what
they perturb and where. Fuzzing generates *inputs*
(random, coverage-guided, or grammar-aware) and watches for crashes, undefined
behavior, or sanitizer trips. [Fault injection](https://quality.stereobooster.com/fault-injection.md)
perturbs the *environment* in a test harness (a failed allocation, a dropped
packet, a crashed dependency) to check the error-handling and fault-tolerance
paths. [Chaos engineering](https://quality.stereobooster.com/chaos-engineering.md) injects the same
kind of environment faults into a *running production* system, verifying
fallbacks under real conditions. Fuzzing probes input handling; fault injection
and chaos probe failure resilience, in a harness and in production respectively.

**Generative testing**

These methods blur together because they all run the code and check the result
without a hand-written expected value. Two roles pull them apart, and a single test
picks one of each.

**Input generators** choose the input, and differ by the steering signal (the
Generative subtree of the [input axis](https://quality.stereobooster.com/input.md)):

- Random: [property-based testing](https://quality.stereobooster.com/property-based-testing.md) samples
  from a generator or schema.
- Coverage feedback: fuzzing mutates inputs steered by
  coverage (raw bytes); [automated test generation](https://quality.stereobooster.com/automated-test-generation.md)
  runs the same feedback loop as a fitness-guided search over structured call sequences.
- Solver: [symbolic execution](https://quality.stereobooster.com/symbolic-execution.md) derives an input
  that reaches a chosen path.
- Systematic: [combinatorial testing](https://quality.stereobooster.com/combinatorial-testing.md) builds a
  covering array over every t-way combination.

**Oracle suppliers** provide the verdict when no expected value is written:

- [Metamorphic testing](https://quality.stereobooster.com/metamorphic-testing.md) checks a relation between
  the outputs of two related inputs.
- [Differential testing](https://quality.stereobooster.com/differential-testing.md) compares against a
  trusted second implementation.

Pick one generator and one oracle: they compose. A coverage-guided fuzzer that checks a
metamorphic relation is fuzzing and metamorphic at once. (The written-answer end, where
the author picks rows and answers by hand, is [example / parameterized
tests](https://quality.stereobooster.com/example-tests.md).)

## Classification

- **Quality dimensions:** Security, Functionality, Reliability.
- **Area:** Parsers, security-sensitive C/C++, kernels and syscall surfaces, network protocols, crypto, compression and image decoders.
- **Guarantee:** Empirical.

## Referenced by

- [Quality dimensions](https://quality.stereobooster.com/quality-dimensions.md) · Quality dimensions
- [Effect scope](https://quality.stereobooster.com/effect.md) · The axes
- [Algorithmic complexity testing](https://quality.stereobooster.com/algorithmic-complexity.md) · Methods
- [Automated test generation](https://quality.stereobooster.com/automated-test-generation.md) · Methods
- [Coverage](https://quality.stereobooster.com/coverage.md) · Methods
- [Differential testing](https://quality.stereobooster.com/differential-testing.md) · Methods
- [Example tests](https://quality.stereobooster.com/example-tests.md) · Methods
- [Penetration testing](https://quality.stereobooster.com/penetration-testing.md) · Methods
- [Schema and boundary validation](https://quality.stereobooster.com/schema-and-boundary-validation.md) · Methods
- [Search-based software testing (SBST)](https://quality.stereobooster.com/search-based-software-testing.md) · Methods
- [Static analysis](https://quality.stereobooster.com/static-analysis.md) · Methods
- [Supply-chain hygiene](https://quality.stereobooster.com/supply-chain-hygiene.md) · Methods
- [Symbolic execution](https://quality.stereobooster.com/symbolic-execution.md) · Methods
- [Temporal-logic falsification](https://quality.stereobooster.com/temporal-logic-falsification.md) · Methods
- [Testing GUI and mobile applications](https://quality.stereobooster.com/testing-gui-and-mobile-apps.md) · Methods
- [Threat modeling](https://quality.stereobooster.com/threat-modeling.md) · Methods
- [Verifying concurrency](https://quality.stereobooster.com/concurrency.md) · Methods
- [Verifying memory safety](https://quality.stereobooster.com/memory.md) · Methods
- [Verifying time and date handling](https://quality.stereobooster.com/time-and-date.md) · Methods
- [How AI fits into software quality](https://quality.stereobooster.com/ai.md) · AI
- [Conventional terminology](https://quality.stereobooster.com/terminology.md) · Conventional
- [Choosing methods](https://quality.stereobooster.com/choosing.md) · Overview

## References

[^rigger2020]: Rigger, Manuel, and Zhendong Su. 2020. "[Testing Database Engines via Pivoted Query Synthesis](https://www.usenix.org/system/files/osdi20-rigger.pdf)." *Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI '20)*. <https://doi.org/10.48550/arXiv.2001.04174>.
[^xu2020]: Xu, Meng, Sanidhya Kashyap, Hanqing Zhao, and Taesoo Kim. 2020. "[Krace: Data Race Fuzzing for Kernel File Systems](https://ieeexplore.ieee.org/ielx7/9144328/9152199/09152693.pdf)." *2020 IEEE Symposium on Security and Privacy (SP)*, 1643–60. <https://doi.org/10.1109/SP40000.2020.00078>.
[^chen2020]: Chen, Hongxu, Shengjian Guo, Yinxing Xue, et al. 2020. "[MUZZ: Thread-aware Grey-box Fuzzing for Effective Bug Hunting in Multithreaded Programs](https://www.usenix.org/system/files/sec20-chen-hongxu.pdf)." *29th USENIX Security Symposium (USENIX Security 20)*, 2325–42. <https://doi.org/10.48550/arXiv.2007.15943>.
[^googlend]: Google. n.d. *[Structure-Aware Fuzzing with libFuzzer](https://github.com/google/fuzzing/blob/master/docs/structure-aware-fuzzing.md)*. Google/fuzzing. <https://github.com/google/fuzzing/blob/master/docs/structure-aware-fuzzing.md>.
[^dutra2021]: Dutra, Rafael, Rahul Gopinath, and Andreas Zeller. 2021. *[FormatFuzzer: Effective Fuzzing of Binary File Formats](https://arxiv.org/pdf/2109.11277)*. <https://doi.org/10.48550/arXiv.2109.11277>.
[^ossfuzz]: Google. 2025. *[OSS-Fuzz: Continuous Fuzzing for Open Source Software](https://github.com/google/oss-fuzz)*. Google/oss-fuzz, GitHub. <https://github.com/google/oss-fuzz>.
[^qu2026]: Qu, Jenny Guanni. 2026. *[Kernel Bugs Hide for 2 Years on Average. Some Hide for 20](https://pebblebed.com/blog/kernel-bugs)*. Pebblebed Blog. <https://pebblebed.com/blog/kernel-bugs>.
[^yang2011]: Yang, Xuejun, Yang Chen, Eric Eide, and John Regehr. 2011. "[Finding and Understanding Bugs in C Compilers](https://users.cs.utah.edu/~regehr/papers/pldi11-preprint.pdf)." *Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '11)*, 283–94. <https://doi.org/10.1145/1993498.1993532>.
[^leroy2009]: Leroy, Xavier. 2009. "[Formal Verification of a Realistic Compiler](https://xavierleroy.org/publi/compcert-CACM.pdf)." *Communications of the ACM* 52 (7): 107–15. <https://doi.org/10.1145/1538788.1538814>.

## Acronyms

- OOM — out of memory
- OSS — open-source software
- PCT — probabilistic concurrency testing
- RNG — random number generator
- RTL — register-transfer level
- SBST — search-based software testing
- SMT — satisfiability modulo theories