Skip to content

Software Quality

Code review

Code review is the practice of developers other than the author reading a change, commenting on it, and approving or blocking it before it lands. The form ranges from Fagan's heavyweight inspection (Fagan 1976)1 to modern pull-request review at Google and Microsoft.

What it catches

The measured value of modern code review is mostly not defect detection (Bacchelli and Bird 2013)2. It is:

  • Knowledge transfer. Participating in review raises a reviewer's exposure to distinct files by 66–150% (Rigby and Bird 2013)3.
  • Alternative solution generation. A reviewer proposes the simpler approach.
  • Gatekeeping for codebase health. Review keeps style, naming and hygiene consistent without every author internalizing every rule.

Most of the defects it does catch are small logical mistakes rather than the design-level problems practitioners expect — a wrong expression in a condition, not a wrong abstraction (Bacchelli and Bird 2013)2.

Code review does not systematically catch the bugs example tests miss — both rely on what the reader (reviewer or test author) has thought of.

Tools and form

Modern code review is tool-based, lightweight, continuous.

  • Forge-native. The forge hosts the review: GitHub pull requests, GitLab merge requests, Bitbucket pull requests, Gerrit (Google-style).
  • CI integration. Linter, formatter, type checker and security scanner all report inline at diff time. Diff-time deployment beats batch (Distefano et al. 2019)4.
  • AI-assisted review. Claude's code-review skill, Cursor, Open Code Review and similar agents extend human reviewers rather than replace them. Claude's skill and Cursor are commercial products. Code Review Bench scores precision (share of comments that flag a real issue) and recall (share of real issues caught) on human-verified pull requests, with a continuously-refreshed online set to keep tools from memorizing the cases.

When to use, when not

Applies to every change. For knowledge spread and codebase health.

Not a primary defect filter. The most effective defect filters are types, static analysis, fuzzing, property-based testing, and tests.

Does not benefit from scaling up. A third reviewer finds no more defects than the first two, and longer reviews and bigger diffs find proportionally fewer.

Evidence

  • Defect finding is not the main outcome. At Microsoft, only a minority of review comments were defect-related — a smaller share of review value than practitioners expect (Bacchelli and Bird 2013)2.
  • Convergent practice across organizations. A multi-site study (Microsoft, Google, AMD, Lucent, 6 OSS projects) found the same shape everywhere: small diffs, 1–2 reviewers, fast turnaround, diminishing returns past 2 reviewers (Rigby and Bird 2013)3.
  • Google scale. Across ~9M reviewed changes, the median change had one reviewer and 70% were merged within 24 hours (Sadowski et al. 2018)5.
  • Defect-finding drops past ~90 min of review, and when the review rate exceeds ~400–500 LOC/hour, in a Cisco/SmartBear study (Cohen 2006)6 that is vendor-published and observational rather than controlled.

Classification

  • Quality dimensions: Maintainability, Functionality.
  • Area: General development; any codebase that goes through pull requests or change reviews.

Referenced by

References

  1. Fagan, Michael E. 1976. "Design and Code Inspections to Reduce Errors in Program Development." IBM Systems Journal 15 (3): 182–211. https://doi.org/10.1147/sj.153.0182. ↩

  2. Bacchelli, Alberto, and Christian Bird. 2013. "Expectations, Outcomes, and Challenges of Modern Code Review." Proceedings of the 35th International Conference on Software Engineering (ICSE '13), 712–21. https://doi.org/10.1109/ICSE.2013.6606617. ↩↩↩

  3. Rigby, Peter C., and Christian Bird. 2013. "Convergent Contemporary Software Peer Review Practices." Proceedings of the 9th Joint Meeting on Foundations of Software Engineering (ESEC/FSE '13), 202–12. https://doi.org/10.1145/2491411.2491444. ↩↩

  4. Distefano, Dino, Manuel Fähndrich, Francesco Logozzo, and Peter W. O'Hearn. 2019. "Scaling Static Analyses at Facebook." Communications of the ACM 62 (8): 62–70. https://doi.org/10.1145/3338112. ↩

  5. Sadowski, Caitlin, Emma Söderberg, Luke Church, Michal Sipko, and Alberto Bacchelli. 2018. "Modern Code Review: A Case Study at Google." Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP '18), 181–90. https://doi.org/10.1145/3183519.3183525. ↩

  6. Cohen, Jason. 2006. Best Kept Secrets of Peer Code Review. SmartBear Software. http://borg.csueastbay.edu/~grewe/CS401/Mat/SWManagement/CodeReview/best-kept-secrets-of-peer-code-review_redirected.pdf. ↩