Skip to content

Software Quality

AI tooling for signals

Coverage, git-history hotspots, and mutation testing are things AI reads to target where to work next, not oracles.

Coverage

LLM test generators use coverage as a fitness signal: CoverUp (Pizzorno and Berger 2025)1 (research), Qodo Cover (open source), Meta TestGen-LLM (Alshahwan et al. 2024)2 (coverage + pass-rate as filters). Search-based tools (EvoSuite, Pynguin, CODAMOSA) are hybridizing with LLMs.

Git-history hotspots

CodeScene (commercial) exposes its hotspot ranking to AI assistants through an MCP server. Without CodeScene, a general-purpose assistant reads Code Maat hotspot output piped to it.

Mutation testing

  • µBERT (Degiovanni and Papadakis 2022)3 — research; masked-LM mutant generation that can be twice as cost-effective as rule-based PITest at the same number of mutants analyzed.
  • LEAM (Tian et al. 2022)4 — a learned (pre-LLM) mutation model.
  • Mutahunter — open-source; mutation testing with LLM-driven mutant generation and survivor analysis.

References

  1. Pizzorno, Juan Altmayer, and Emery D. Berger. 2025. "CoverUp: Effective High Coverage Test Generation for Python." Proceedings of the ACM on Software Engineering, 2897–919. https://doi.org/10.1145/3729398. ↩

  2. Alshahwan, Nadia, Jubin Chheda, Anastasia Finegenova, et al. 2024. "Automated Unit Test Improvement Using Large Language Models at Meta." Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering (FSE 2024), 185–96. https://doi.org/10.1145/3663529.3663839. ↩

  3. Degiovanni, Renzo, and Mike Papadakis. 2022. "μBERT: Mutation Testing using Pre-Trained Language Models." 2022 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 160–69. https://doi.org/10.1109/icstw55395.2022.00039. ↩

  4. Tian, Zhao, Junjie Chen, Qihao Zhu, Junjie Yang, and Lingming Zhang. 2022. "Learning to Construct Better Mutation Faults." Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (ASE), 1–13. https://doi.org/10.1145/3551349.3556949. ↩