Skip to content

Software Quality

AI tooling for signals

Coverage, git-history hotspots, and mutation testing are things AI reads to target where to work next, not oracles. The field moves quickly, and these entries may be out of date.

Coverage

LLM test generators use coverage as a fitness signal: CoverUp (Pizzorno and Berger 2025)1 (research), Qodo Cover (open source), Meta TestGen-LLM (Alshahwan et al. 2024)2 (coverage + pass-rate as filters). Search-based tools (EvoSuite, Pynguin, CODAMOSA) are hybridizing with LLMs.

Git-history hotspots

No dedicated tool pairs hotspot detection with an LLM. The gap is one a general-purpose agent covers: Code Maat output piped through an assistant.

Mutation testing

  • µBERT (Degiovanni and Papadakis 2022)3 — research; masked-LM mutant generation that can be twice as cost-effective as rule-based PITest at the same number of mutants analyzed.
  • LEAM (Tian et al. 2022)4 — a learned (pre-LLM) mutation model.
  • Mutahunter — open-source; mutation testing with LLM-driven mutant generation and survivor analysis.

References

  1. Pizzorno, Juan Altmayer, and Emery D. Berger. 2025. "CoverUp: Effective High Coverage Test Generation for Python." Proceedings of the ACM on Software Engineering, 2897–919. https://doi.org/10.1145/3729398. ↩

  2. Alshahwan, Nadia, Jubin Chheda, Anastasia Finegenova, et al. 2024. "Automated Unit Test Improvement Using Large Language Models at Meta." Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering (FSE 2024), 185–96. https://doi.org/10.1145/3663529.3663839. ↩

  3. Degiovanni, Renzo, and Mike Papadakis. 2022. "μBERT: Mutation Testing using Pre-Trained Language Models." 2022 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 160–69. https://doi.org/10.1109/icstw55395.2022.00039. ↩

  4. Tian, Zhao, Junjie Chen, Qihao Zhu, Junjie Yang, and Lingming Zhang. 2022. "Learning to Construct Better Mutation Faults." Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (ASE), 1–13. https://doi.org/10.1145/3551349.3556949. ↩