Skip to content

Software Quality

Static analysis

Reading the source without running it, from cheap linters to whole-program dataflow.

Static analysis examines source code without running it and reports findings. The family ranges from cheap pattern-matching linters to heavyweight whole-program dataflow analyzers.

Every method here works the same way: it checks the code against a rule or invariant. Most report likely problems with high confidence rather than a proof; the exceptions are the checks decided outright over a single procedure's control flow or dataflow, where the finding is a fact about the code. What separates the sub-families is how deep they look and what they look for.

The sub-families

  • Linters: per-language, pattern- and rule-based bug finders run at diff time. Off-by-one, missing await, suspicious cast, API misuse. Fast, high-volume, tuned to keep false positives low.
  • Deep static analysis: cross-language query and dataflow engines (CodeQL, Semgrep), and automaton analysis of regular expressions. Builds a model of the program and queries it, rather than matching a pattern in one file.
  • Secret scanning: rule- and entropy-matching over unparsed text, reaching the whole git history rather than only the current tree.
  • Dead-code detection: code no execution can reach — unused files, exports, and dependencies from the call graph, plus unreachable statements, branches no input can satisfy, and values assigned and never read.
  • Clone detection: near-duplicate fragments across the codebase, above a tuned similarity threshold.
  • Database migration safety: linting schema migrations for locks and backward-incompatible changes that break a rolling deploy.

What the family catches

Each sub-page covers its own bug classes; several defect classes are themselves whole sub-families with dedicated pages:

  • Memory-safety bugs. Buffer overflow, use-after-free, double-free, null deref. Tools targeting C/C++ are strongest here; the depth argument is on memory safety.
  • Concurrency bugs. Data races, deadlocks, missing locks. Facebook's Infer checks for them; the method sits on concurrency.
  • Security taint. User input reaching a dangerous sink (SQL, shell, eval, file path). Interprocedural source→sink dataflow is a distinct sub-family: taint analysis.
  • Committed secrets. Credentials in the tree or anywhere in history: secret scanning.
  • ReDoS and API misuse. Statically detectable, covered under deep static analysis and linters.
  • Deadness and duplication. Code nothing can reach, and code that exists twice: dead-code detection and clone detection.

When to use, when not

Use:

  • Diff-time deployment, posting results inline as PR comments.
  • High signal-to-noise rule selection. A small, high-confidence initial ruleset is easier to expand than to cut back.
  • Memory-safety-sensitive C/C++ code, where the heavyweight analyzers earn their cost.

Don't:

  • Treat static analysis as a complete verification toolkit. It catches patterns and reachable dataflow; it misses logic. Generative methods (property tests, fuzzing) and code review handle what it can't.
  • Drown developers in warnings. Across studies the deciding factor for adoption is the false-positive rate and whether developers trust the output, not the algorithm (Bessey et al. 2010; Li et al. 2024)1 2.

Evidence

How much of what these tools report gets acted on — the adoption economics — has consistent answers across four decades of studies, and they hold for every method in the family:

  • Coverity at scale. A retrospective across ~700 customer codebases found the false-positive rate and whether developers understand and trust the output to be the deciding factors for adoption (Bessey et al. 2010)1.
  • Facebook Infer and Zoncolan. Diff-time deployment jumped the fix rate from near-zero (batch) to over 70%; developers have resolved over 100,000 Infer-flagged issues since 2014, across codebases of tens of millions of lines (Distefano et al. 2019)3.
  • FindBugs at Google. In the May 2009 company-wide fixit, 282 engineers filed 10,479 reviews across 3,954 of 9,473 open issues, over 77% of reviews classified the warning as a real defect worth fixing, and submitted changes removed more than 1,000 issues (Ayewah and Pugh 2010)4. Reviewers chose what to review and concentrated on the Correctness category — 71% of those issues reviewed against 17% of the rest — so 77% is a rate over self-selected warnings, not over all warnings.

How much of what exists gets reported, and what those reports were worth, answer far less well:

  • Yield is not recall. A count of issues actioned measures precision times scale times time, not the share of real bugs a tool finds. Against fixed bug sets, three detectors found 4.5% of 594 real Java bugs (Habib and Pradel 2018)5 and six C/C++ analyzers missed 47–80% of 192 real vulnerabilities (Lipp et al. 2022)6. A low-recall analyzer run continuously over an enormous codebase still accumulates six figures of fixes. Both results hold; neither substitutes for the other.
  • Measured recall is a floor, not an estimate. The bug sets are drawn from defects committed to version control, so any bug a detector caught before check-in is missing from the sample by construction — a caveat the authors state themselves (Habib and Pradel 2018)5. What a diff-time deployment prevents is therefore higher than the measured figure, by an unmeasured amount.
  • A real defect is not a production incident. In the Google fixit, none of the serious bugs turned out to be associated with serious incorrect behavior in production: they sat in code not yet pushed, or not executed, or executed in conditions that misbehaved only subtly — performance degradation and the like (Ayewah and Pugh 2010)4. The authors attribute this to Google's testing and monitoring catching those problems by other means rather than to any failure of the analysis, and put the value of FindBugs in finding them earlier, where remediation is cheap. Static analysis competing against strong testing buys time, which is a weaker and more contingent claim than catching what nothing else would.

The per-method numbers, including the security-specific ones, are on deep static analysis and secret scanning.

Referenced by

References

  1. Bessey, Al, Ken Block, Ben Chelf, et al. 2010. "A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World." Communications of the ACM 53 (2): 66–75. https://doi.org/10.1145/1646353.1646374. ↩↩

  2. Li, Chunmiao, Yijun Yu, Haitao Wu, Yuanliang Zhang, Zhi Jin, and Zhenjiang Hu. 2024. "Unleashing the Power of Clippy in Real-World Rust Projects." Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings, 318–19. https://doi.org/10.1145/3639478.3643096. ↩

  3. Distefano, Dino, Manuel Fähndrich, Francesco Logozzo, and Peter W. O'Hearn. 2019. "Scaling Static Analyses at Facebook." Communications of the ACM 62 (8): 62–70. https://doi.org/10.1145/3338112. ↩

  4. Ayewah, Nathaniel, and William Pugh. 2010. "The Google FindBugs Fixit." Proceedings of the 19th International Symposium on Software Testing and Analysis (ISSTA '10) (New York), 241–52. https://doi.org/10.1145/1831708.1831738. ↩↩

  5. Habib, Andrew, and Michael Pradel. 2018. "How Many of All Bugs Do We Find? A Study of Static Bug Detectors." Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE '18), 317–28. https://doi.org/10.1145/3238147.3238213. ↩↩

  6. Lipp, Stephan, Sebastian Banescu, and Alexander Pretschner. 2022. "An Empirical Study on the Effectiveness of Static C Code Analyzers for Vulnerability Detection." Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2022), 544–55. https://doi.org/10.1145/3533767.3534380. ↩