Skip to content

Software Quality

AI tooling for requirements engineering

AI tooling for requirements engineering applies an LLM in two ways: gathering produces requirements from informal input, and detection audits existing ones. This is the AI as the oracle branch of the section spine: no sound oracle exists yet, so AI judges intent rather than writing or consuming a check.

The shape of the field

  1. Gathering has usable tools; cross-requirement detection is mostly research. Most shipping detection tools score individual requirements and skip cross-requirement contradiction.
  2. Spec-driven AI development is now a workflow category. A spec-first scaffold orders the AI's work; it does not check the spec against a quality rule set.

The two activities

Gathering — LLM produces requirements. Informal input (a Slack thread, an interview transcript, a regulation, a brief) becomes structured output (user stories, acceptance criteria, PRDs). The LLM produces requirement-shaped text; whether it describes what the team or user actually wants is a separate question AI doesn't answer.

Detection — LLM audits requirements. A set of existing requirements becomes a list of problems, the same audit a human reviewer does by hand (Requirements quality checking). It decomposes into sub-tasks of decreasing maturity: per-requirement quality scoring, ambiguity detection, and cross-requirement contradiction.

Tools for gathering

GitHub Spec Kit imposes spec-first discipline on AI-assisted development and works across agent hosts; SpecStory captures AI design conversations into decisions.md artifacts, close to an architecture decision record. ChatPRD is a commercial hosted PRD-drafting tool. Commercial RM platforms (Jama, IBM DOORS Next + watsonx, Polarion, Codebeamer) add AI authoring that scores against INCOSE/EARS; commercial product-management suites (Productboard, Atlassian Rovo, Linear) draft tickets and cluster feedback upstream of a PRD.

Generating Gherkin acceptance test scenarios from user stories has been measured at one company (Ferreira et al. 2025)1: over 166 uses, 95% of the 65 responses collected called the scenarios helpful, and the executable test scripts derived from them were accurate and complete 60% of the time, or 92% after minor fixes or extra context.

Tools for inconsistency / quality detection

  • QuARS (ISTI-CNR) — lexical and syntactic rule matching over requirement text, pre-2010.
  • ALICE (Gärtner and Göhlich 2024)2 — formal logic plus an LLM, classifying contradictions in requirements written in controlled natural language; on one of the paper's datasets it reached 60% recall where the LLM-only method reached none.
  • LMAdetect (Gao et al. 2026)3 — combines heuristic rules with an LLM to detect ambiguity, and scored a higher F1 than both a rules-only and an LLM-only method.
  • Lubos et al. (Lubos et al. 2024)4 — an LLM scored requirements against ISO/IEC/IEEE 29148; its agreement with software engineers was weak or absent when they judged independently, and rose once they saw the LLM's verdict and explanation.
  • SpecFix (Jia et al. 2025)5 — repairs an ambiguous NL problem description by aligning it with the input/output examples and reducing the model's code-generation uncertainty, measured on programming-benchmark descriptions rather than on requirements documents.
  • Knowledgebase Guardian (dataroots) — a small open-source reference implementation of RAG + LLM contradiction detection across a corpus, and a starting point for a custom checker.

Polarion's Consistency Check, in the paid Polarion Copilot add-on, compares linked work items against each other for contradictions rather than scoring each requirement alone; its evidence is vendor-reported.

Where existing tooling stops

  • Cross-requirement contradiction detection at scale. The logical-form + LLM hybrids are research prototypes, and no shipping tool reproduces them.
  • NFR conflict detection. Conflicts among latency, throughput, security, and cost are under-supported; pre-LLM matrix-map approaches haven't been folded in.
  • Spec ↔ regulation tracing. Bidirectional drift detection between requirements and a regulatory clause (GDPR, ISO 26262, DO-178C) is largely manual.
  • Transcript → structured requirement with provenance. Enterprise assistants do the recap; none emit a structured, traceable artifact.
  • Open-source agent skill for requirements quality. No established SKILL.md package targets ISO/IEC/IEEE 29148 or INCOSE checking.

Tools vs research

A 2025 systematic review of 74 studies published in 2023–2024 (Zadenoori et al. 2025)6 found elicitation and validation the most-studied RE tasks, GPT models in 57 of the 74, zero-shot and few-shot prompting the dominant strategies, and 56 evaluated by laboratory experiment rather than in the field.

Referenced by

References

  1. Ferreira, Margarida, Luis Viegas, Joao Pascoal Faria, and Bruno Lima. 2025. "Acceptance Test Generation with Large Language Models: An Industrial Case Study." 2025 IEEE/ACM International Conference on Automation of Software Test (AST), 1–11. https://doi.org/10.1109/ast66626.2025.00007. ↩

  2. Gärtner, Alexander Elenga, and Dietmar Göhlich. 2024. "Automated Requirement Contradiction Detection through Formal Logic and LLMs." Automated Software Engineering 31. https://doi.org/10.1007/s10515-024-00452-x. ↩

  3. Gao, Juntao, Fang Liu, and Yilong Yang. 2026. "Requirement Ambiguity Detection Method Based on Large Language Model." Journal of Computer Research and Development 63 (4): 827–53. https://doi.org/10.7544/issn1000-1239.202550700. ↩

  4. Lubos, Sebastian, Alexander Felfernig, Thi Ngoc Trang Tran, et al. 2024. "Leveraging LLMs for the Quality Assurance of Software Requirements." Proceedings of the 32nd IEEE International Requirements Engineering Conference (RE '24), 389–97. https://doi.org/10.1109/RE59067.2024.00046. ↩

  5. Jia, Haoxiang, Robbie Morris, He Ye, Federica Sarro, and Sergey Mechtaev. 2025. "Automated Repair of Ambiguous Problem Descriptions for LLM-Based Code Generation." Proceedings of the 40th IEEE/ACM International Conference on Automated Software Engineering (ASE '25), 367–79. https://doi.org/10.1109/ASE63991.2025.00038. ↩

  6. Zadenoori, Mohammad Amin, Jacek Dąbrowski, Waad Alhoshan, Liping Zhao, and Alessio Ferrari. 2025. Large Language Models (LLMs) for Requirements Engineering (RE): A Systematic Literature Review. https://doi.org/10.48550/arXiv.2509.11446. ↩