# Bugs in AI-generated code AI-generated code carries a characteristic failure mode: output that compiles and reads as correct but does the wrong thing. As models write more of a codebase, that defect profile becomes the case verification has to catch. A 2025 systematic review of the defect literature (Gao et al. 2025; Gao 2025)[^gao2025] [^gao2025icsme] supplies the taxonomy and the distribution. It is one recent study, not yet independently replicated at scale, and the structural account is the more settled part. ## What the defects look like The review sorts reported bugs into eight categories (functional, reliability, syntax, code style, hallucination, system, test, and unspecified) and finds two dominate: **functional** bugs (the code does the wrong thing), in 78% of the surveyed studies, and **syntax** bugs, in 42% (Gao et al. 2025)[^gao2025]. Within functional bugs, the largest subcategories are semantic and logic bugs (26%) and bugs in variables, objects, parameters and properties (27%). Semantic and logic bugs **do not break compilation** — the program runs and returns an answer; the answer is wrong. Syntax errors, by contrast, are the cheap case: a compiler rejects them at once. ## Why it happens The review attributes the pattern to two structural causes rather than a passing model weakness (Gao et al. 2025; Gao 2025)[^gao2025] [^gao2025icsme]: - **The training corpus carries the defects.** Models learn from public code, which itself contains bugs and uneven quality, so some of what they reproduce is wrong by inheritance. Strengths also track the corpus — a model is most reliable in the languages it saw most of. - **Generation has no feedback.** A model emits code without executing it or running a test against it, so nothing during generation catches a syntactically-correct-but-semantically-wrong result. Bugs appear in every model family the review surveyed; larger, better instruction-following models shift the distribution without removing it. ## Why it makes the oracle matter A sound oracle in the loop (a compiler, a test suite, a sanitizer) supplies the feedback generation lacks, and that is what makes the [consume relationship](https://quality.stereobooster.com/ai.md#ai-consumes-an-oracle) the strong one. The review reports the field moving the same way, from relying on static analysis alone toward "multi-layered intervention" combining program analysis, prompt engineering, and coding agents that generate, test, and debug code (Gao et al. 2025)[^gao2025]. The same defect profile explains why the [judge relationship](https://quality.stereobooster.com/ai.md#ai-is-the-oracle) is the weak one. Semantic and logic bugs are what an AI reviewer with no ground truth is worst at: the code reads as correct, and only running it (or knowing the intent it was meant to satisfy) reveals otherwise. ## Referenced by - [Agent experience](https://quality.stereobooster.com/ai-agent-experience.md) · AI - [How AI fits into software quality](https://quality.stereobooster.com/ai.md) · AI ## References [^gao2025]: Gao, Ruofan, Amjed Tahir, Peng Liang, Teo Susnjak, and Foutse Khomh. 2025. *[A Survey of Bugs in AI-Generated Code](https://arxiv.org/pdf/2512.05239)*. . [^gao2025icsme]: Gao, Ruofan. 2025. "[Bugs in AI-Generated Code: Understanding Bug Patterns and Possible Fix Strategies](https://doi.org/10.1109/ICSME64153.2025.00098)." *2025 IEEE International Conference on Software Maintenance and Evolution (ICSME)*, 881–83. .