# Agent security testing Agent security testing checks whether attacker-controlled text can make an agent call a tool the user did not authorize. The oracle is the tool-call trace, inspected for that action. The exposure is ordinary: an agent reads content it does not control and holds tools that can spend money, write to a repository, or send mail. The text reaches the model through two channels. It arrives as **data** — a fetched page, a repository file, an issue comment, the result of another tool — which the agent was asked to read and which it may instead obey. Or it arrives as a **tool definition** — the names and descriptions a skill or an [MCP](https://modelcontextprotocol.io/) server supplies, which the agent loads into its own context as instructions. Testing the first channel means running the agent, because whether the text is obeyed depends on a sampled model. Testing the second means reading the declared surface, which is text an attacker wrote and nothing has executed. Both channels pose the [taint problem](https://quality.stereobooster.com/taint-analysis.md) with a stochastic sink: untrusted data reaches a privileged operation, and what decides whether it reaches is a sampled model rather than a program path, so dataflow over the source cannot settle it. The check is whether a named tool was called with particular arguments — a predicate over a recorded trace, decided mechanically — not whether the agent's answer was good, which would need a judgment oracle. That is what makes this testable in the ordinary sense. The targets differ in what can be done to them. A harness runs, so its trace is available. A skill is a package, inspected before it is installed. An MCP server is a live surface that can change after it is approved. ## What it catches - **Instruction-carrying data reaching a privileged tool.** The injected text is read as an instruction rather than as content, and the agent calls a tool on the attacker's behalf. - **Exfiltration through an authorized channel.** Every individual step is permitted — read the file, compose the message, send it — and the composition leaks. An allowlist over tool *names* does not catch it; the argument values carry the data out. - **The gap between the stated policy and the enforced one.** A confirmation gate turns out to accept a rephrasing, a sandbox turns out to have an outbound proxy, a denylist turns out to be routed around by a tool's own arguments. - **A poisoned tool definition.** The instruction arrives in the description rather than in the data, and one server's description can shadow another server's tool. ## What it cannot catch Running the agent measures one corpus, one model version, and one system prompt, and all three change without notice. Because the model is sampled, a single clean run carries almost no information: the measurement is a rate over repeats, and a zero attack-success rate means the payloads in hand did not work. Reading the surface instead of running it removes the sampling problem and replaces it with a narrower one — a pattern match catches the phrasings someone wrote a rule for. Neither form bounds what an attacker who has not been thought of can do. ## Testing an agent harness The harness is the composed system: its tool surface, its sandbox, its confirmation gates. The model is a component of it, not the subject, and two harnesses wrapping the same model produce different rates. ### Adversarial corpora A corpus supplies attacker goals, injection points, and the ordinary user tasks the agent is running when the injection arrives. InjecAgent holds 1,054 test cases over 17 user tools and 62 attacker tools, sorted into two attacker intentions: direct harm to the user, and exfiltration of private data (Zhan et al. 2024)[^zhan2024injecagent]. AgentDojo holds 97 user tasks and 629 security test cases across four stateful environments — a workspace, a travel agent, a banking app, a Slack workspace — with 70 tools between them (Debenedetti et al. 2024)[^debenedetti2024agentdojo]. The score is an attack success rate: the fraction of security cases in which the attacker's goal was met. The goal is written as a concrete tool call before the run starts, so the verdict needs no interpretation. A second rate, the fraction of cases where the agent still completed the user's own task, separates a harness that resists attacks from one its defenses have made useless. The payloads are published and they age, so a defense tuned until the corpus is clean has been fitted to that corpus. ### Adaptive attack A fixed suite scores the defense against attacks written before the defense existed. AgentDojo is built as an extensible environment rather than a static test suite for this reason, and it reports a metric that models an adaptive attacker directly: over a collection of attacks, a security case counts as broken if any one of them breaks it. On GPT-4o, its baseline injection reached a targeted attack success rate of 45.8% naming the model and the user generically, and 23.2% when it guessed the user's name wrongly (Debenedetti et al. 2024)[^debenedetti2024agentdojo]. A corpus score is the rate for an attacker who ran no reconnaissance. The human form is [penetration testing](https://quality.stereobooster.com/penetration-testing.md) pointed at the agent, bounded by the tester rather than by a corpus. ## Testing a skill A skill is a text artifact — instructions, tool descriptions, and whatever files ship beside them. Installing it is what puts that text into the agent's context, so it is attacked without ever running. A scan reads the package instead of executing it, so it costs a fetch. A scanner that judges with a model inherits that model's own susceptibility to the text it is reading. ## Testing an MCP server An MCP server carries the same poisoned-description risk as a skill, and adds two of its own. A server sits alongside others, so a description can reference another server's tool and pull calls toward itself — an attack that is invisible in any single server read on its own. And the surface is live rather than packaged: a server declares whether its tool list can change and emits a notification when it does, so the tools approved at install are not necessarily the tools in context a week later. A scan of an MCP server is a point-in-time result about a surface its operator controls. The specification states that descriptions of tool behavior should be treated as untrusted unless the server is trusted, and specifies no mitigation, which leaves the check to the host. ## Restriction is not a test Testing measures residual risk; it does not reduce it. The defenses that reduce it are structural — an allowlist of tools per task, a sandbox with no outbound network, human confirmation on irreversible actions, and information-flow control over tool arguments rather than tool names. CaMeL extracts the control and data flow from the trusted user query and runs it in an interpreter that tracks provenance and enforces policy at the point a tool is called, so untrusted data cannot influence which tools run (Debenedetti et al. 2025)[^debenedetti2025defeating]. The guarantee comes from the construction and holds whether or not the model is fooled. ## Tools - **Agent harnesses** — [AgentDojo](https://github.com/ethz-spylab/agentdojo) (stateful environments, adaptive attacks, and a utility-under-attack metric alongside the security one) and [InjecAgent](https://github.com/uiuc-kang-lab/InjecAgent) (tool-integrated agents, user-harm and exfiltration intents). - **Skills** — [SkillSpector](https://github.com/NVIDIA/SkillSpector) reads a directory or archive with pattern and syntax-tree rules and returns a risk score, staying local under `--no-llm`. - **MCP servers** — [MCP Scanner](https://github.com/cisco-ai-defense/mcp-scanner) pulls a server's tool descriptions, prompts, and resources and matches them against signature rules, with the model judge and the hosted analyzer as optional flags rather than the analysis. Snyk's Agent Scan covers skills and servers both, but its analysis runs behind an account token that sends tool names and descriptions to a hosted API. - **Model endpoints** — [garak](https://github.com/NVIDIA/garak) probes a model with a library of attack classes and reports which land; [promptfoo](https://github.com/promptfoo/promptfoo) generates application-specific red-team cases from a description of the system and runs them in CI; [PyRIT](https://github.com/Azure/PyRIT) scripts multi-turn attacks against a target and scores the transcripts. These reach a model, not an agent: whether the sandbox holds and whether a confirmation gate can be talked past need cases written against the deployed tool surface. - **Checklists** — the [OWASP Top 10 for LLM Applications](https://genai.owasp.org/llm-top-10/) enumerates the failure classes to write cases against, including prompt injection and excessive agency. It is a checklist, not a measurement. ## When to use, when not **Use:** - Before shipping any agent that reads attacker-influenced content and holds a tool with an irreversible effect. Fetching a URL, reading a repository, and processing issue comments all qualify. - Before installing a skill or connecting an MCP server, where a surface scan is the only form cheap enough to gate every install. - After any change to the tool surface, the system prompt, or the model version, each of which invalidates the previous measurement. - After adding a restriction, where the run establishes that the allowlist or the sandbox does what its author believes. **Don't:** - As the reason to grant a capability. A rate measured against known payloads is not a basis for handing an agent an irreversible tool; the capability decision comes from [threat modeling](https://quality.stereobooster.com/threat-modeling.md) and is enforced by construction. - On a single run. One clean trace over a nondeterministic system is not evidence. - To evaluate the model. Robustness as a model property is a different question from whether this agent can be made to act. ## Evidence InjecAgent puts a ReAct-prompted GPT-4 agent at a 24% attack success rate, rising to 47% when the injected instruction is reinforced with a hacking prompt (Zhan et al. 2024)[^zhan2024injecagent]. On AgentDojo the more capable models are the easier targets, an inverse scaling relation the authors attribute to weaker models failing to carry out the attacker's goal even when the injection lands (Debenedetti et al. 2024)[^debenedetti2024agentdojo]. CaMeL solves 77% of AgentDojo's user tasks while guaranteeing no policy violation, against 84% for the same tasks undefended (Debenedetti et al. 2025)[^debenedetti2025defeating]. Whether a corpus result predicts anything about a deployed agent is unmeasured. No study takes an agent scored on a published corpus and reports what an attacker with reconnaissance achieves against it, and no longitudinal data exists on how fast a score decays as payloads circulate. The published rates for poisoned skills and MCP servers in the wild come from scanner vendors rather than from independent replication. ## Classification - **Quality dimensions:** Security. - **Area:** Agents that read attacker-influenced content while holding tools that act — a harness under an operator's control, a skill before installation, an MCP server before connection; the content arriving as fetched pages, repository files, issue comments, or as the tool definitions a third-party supplies. - **Guarantee:** Heuristic, Empirical (injection-corpus). ## Referenced by - [How AI fits into software quality](https://quality.stereobooster.com/ai.md) · AI ## References [^zhan2024injecagent]: Zhan, Qiusi, Zhixiang Liang, Zifan Ying, and Daniel Kang. 2024. "[InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://aclanthology.org/2024.findings-acl.624.pdf)." *Findings of the Association for Computational Linguistics: ACL 2024*, 10471–506. . [^debenedetti2024agentdojo]: Debenedetti, Edoardo, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. 2024. "[AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents](https://arxiv.org/pdf/2406.13352)." *Advances in Neural Information Processing Systems 38 (NeurIPS 2024) Track on Datasets and Benchmarks*, 82895–920. . [^debenedetti2025defeating]: Debenedetti, Edoardo, Ilia Shumailov, Tianqi Fan, et al. 2025. "[Defeating Prompt Injections by Design](https://arxiv.org/pdf/2503.18813)." .