Skip to content

Software Quality

Code review

Code review is the practice of developers other than the author reading a change, commenting on it, and approving or blocking it before it lands. The form ranges from Fagan's heavyweight inspection (Fagan 1976)1 to modern pull-request review at Google and Microsoft.

What it catches

The measured value of modern code review is mostly not defect detection (Bacchelli and Bird 2013)2. It is:

  • Knowledge transfer. Participating in review raises a reviewer's exposure to distinct files by 66–150% (Rigby and Bird 2013)3.
  • Alternative solution generation. A reviewer proposes the simpler approach.
  • Gatekeeping for codebase health. Style, naming, hygiene consistency without needing every author to internalize every rule.

The defects review does catch are usually high-level ones — wrong abstraction, missing edge case, inconsistency with other parts of the codebase — that automated checkers miss.

Code review does not systematically catch the bugs example tests miss — both rely on what the reader (reviewer or test author) has thought of.

Tools and form

Modern code review is tool-based, lightweight, continuous. Tools:

  • Forge-native. GitHub pull requests, GitLab merge requests, Bitbucket pull requests, Gerrit (Google-style).
  • CI integration. Linter, formatter, type checker, security scanner all reporting inline at diff time. Diff-time deployment beats batch (Distefano et al. 2019)4, 70% fix rate vs ~zero.
  • AI-assisted review. Claude's code-review skill, Cursor, Open Code Review, and similar agents. These extend human reviewers rather than replace them. Code Review Bench scores precision (share of comments that flag a real issue) and recall (share of real issues caught) on human-verified pull requests, with a continuously-refreshed online set to keep tools from memorizing the cases.

When to use, when not

Applies to every change. For knowledge spread, codebase health, and the kind of high-level defect that automated tools miss.

Not a primary defect filter. The strongest defect filters are types, static analysis, fuzzing, property-based testing, and tests.

Does not benefit from scaling up. More reviewers, longer reviews, bigger diffs all reduce effectiveness.

Evidence

  • Modern review is mostly knowledge transfer. At Microsoft, only a minority of review comments were defect-related — a smaller share of review value than practitioners expect (Bacchelli and Bird 2013)2.
  • Convergent practice across organizations. A multi-site study (Microsoft, Google, AMD, Lucent, 6 OSS projects) found the same shape everywhere: small diffs, 1–2 reviewers, fast turnaround, diminishing returns past 2 reviewers (Rigby and Bird 2013)3.
  • Google scale. Over ~9M reviewed changes, median 1 reviewer, 70% merged within 24h (Sadowski et al. 2018)5.
  • Defect-finding drops past ~90 min of review, and when the review rate exceeds ~400–500 LOC/hour, in a Cisco/SmartBear study (Cohen 2006)6; vendor-published — treat as directionally suggestive.

Classification

  • Quality dimensions: Maintainability, Functionality.
  • Area: General development; any codebase that goes through pull requests or change reviews.

Referenced by

References

  1. Fagan, Michael E. 1976. "Design and Code Inspections to Reduce Errors in Program Development." IBM Systems Journal 15 (3): 182–211. https://doi.org/10.1147/sj.153.0182. ↩

  2. Bacchelli, Alberto, and Christian Bird. 2013. "Expectations, Outcomes, and Challenges of Modern Code Review." Proceedings of the 35th International Conference on Software Engineering (ICSE '13), 712–21. https://doi.org/10.1109/ICSE.2013.6606617. ↩↩

  3. Rigby, Peter C., and Christian Bird. 2013. "Convergent Contemporary Software Peer Review Practices." Proceedings of the 9th Joint Meeting on Foundations of Software Engineering (ESEC/FSE '13), 202–12. https://doi.org/10.1145/2491411.2491444. ↩↩

  4. Distefano, Dino, Manuel Fähndrich, Francesco Logozzo, and Peter W. O'Hearn. 2019. "Scaling Static Analyses at Facebook." Communications of the ACM 62 (8): 62–70. https://doi.org/10.1145/3338112. ↩

  5. Sadowski, Caitlin, Emma Söderberg, Luke Church, Michal Sipko, and Alberto Bacchelli. 2018. "Modern Code Review: A Case Study at Google." Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP '18), 181–90. https://doi.org/10.1145/3183519.3183525. ↩

  6. Cohen, Jason. 2006. Best Kept Secrets of Peer Code Review. SmartBear Software. http://borg.csueastbay.edu/~grewe/CS401/Mat/SWManagement/CodeReview/best-kept-secrets-of-peer-code-review_redirected.pdf. ↩