AgentDojo
A benchmark environment for testing whether a large language model (LLM) agent completes its task while resisting prompt-injection attacks in tool outputs and external content.
In our briefings
The claim each of the newest briefings on AgentDojo would have you leave with.
A delegated mission's authority must be a subset of the parent's, McGuinness argues, and OAuth token exchange (RFC 8693) does not guarantee it. Token exchange carries the delegation, but he calls enforcing the narrowing implementation-dependent and not a protocol guarantee. He writes that in production token exchange deployments today, child tokens routinely inherit the parent's full scope unless the implementation explicitly narrows them, so agents compound a failure that OAuth delegation already has. A child agent may also run in a different runtime or expose weaker telemetry, so authority that is narrower on paper can be less governable in practice. He proposes that the mission's state owner rate each hop's trust quality, meaning how well that hop can be observed and checked, and give a low-rated child a more conservative budget. For a sub-agent an orchestrator spawns at runtime, he says the state owner must still issue its authority, but asks what triggers that issuance. Our briefing on Uber's production design found a token chain that records each agent hop without describing any narrowing. Our briefing on Highflame's open-source ZeroID found narrowing by scope intersection offered as an explicit step.
OAuth token exchange does not guarantee a sub-agent stays inside its parent's accessBI-FIND-2026-0228
OWASP's AIVSS v0.8 starts from a CVSS v4.0 base score and adds an uplift worth part of the gap between that base and 10. Ten agentic factors, each scored 0, 0.5 or 1, set the share, and an exploit-maturity multiplier scales it, by default 0.97. With the default mitigation factor of 1.0 (no or weak mitigation), all ten worked examples in the document land between 7.1 and 10.0, seven Critical and three High, and a goal-manipulation flaw with a CVSS base of 2.1 comes out at 7.1. The authors read a jump like that as a sign that the agentic deployment context, not the technical flaw, is the main risk driver. On our reading of the formula, a score can only fall below its CVSS base through the mitigation factor, and the document warns that AIVSS scores are not directly comparable to CVSS base scores.
OWASP's AIVSS lifts a CVSS 2.1 agent flaw to 7.1, and all ten worked examples land High or CriticalBI-FIND-2026-2299
On AgentDyn, the authors' benchmark of 60 open-ended tasks that make the agent replan as it goes, none of the nine defenses they ran on GPT-4o both kept the agent working and stopped attacks. Prompt Sandwiching, Spotlighting and the PromptGuard2 detector finished 55% to 63% of tasks with no attack, at or above the undefended agent's 53.3%, but left 27% to 31% of attacks succeeding. PromptGuard2 kept that work only when no attack came: under attack it finished 20.80% of tasks, because it discards a tool's whole output once it detects an injection. The ProtectAI and PIGuard detectors and the Tool Filter, CaMeL and Progent system defenses held attack success to 4.2% or less and finished 10% of tasks or fewer. DRIFT, built by three of the authors, finished the most tasks of that group with no attack, 30.0%, with 0.8% attack success, and CaMeL finished no task on any of the 12 models tested. The authors' own verdict is that none of them attain "acceptable performance for real-world deployment on AgentDyn".
None of nine prompt-injection defenses both stopped attacks and kept a GPT-4o agent working on tasks that change mid-runBI-FIND-2026-1854
›Cited in 30 briefings
Every briefing that discusses this term, newest first.
- OAuth token exchange does not guarantee a sub-agent stays inside its parent's access
Article · · Control Plane by Karl McGuinness · Karl McGuinness · Mar 18, 2026
- OWASP's AIVSS lifts a CVSS 2.1 agent flaw to 7.1, and all ten worked examples land High or Critical
Article · · OWASP AIVSS project · Ken Huang et al. · Oct 22, 2026
- None of nine prompt-injection defenses both stopped attacks and kept a GPT-4o agent working on tasks that change mid-run
Paper · · arXiv · Hao Li et al. · Feb 3, 2026
- Each agent and each tool call can pass its check while the system fails, an Oxford-led survey argues
Paper · · arXiv · Christian Schroeder de Witt · Apr 29, 2026
- AgentArmor cut attack success to 3% in its authors' tests; most misses came when a user said to follow a file
Paper · · arXiv · Peiran Wang et al. · Nov 18, 2025
- A fixed tool plan blocks prompt injection but breaks tasks; in DRIFT's tests, letting a model widen it wins most back
Paper · · arXiv · Hao Li et al. · Mar 26, 2026
- With real MCP tools for cloud, database and email, most injected attacks got past four models, even with a plan
Paper · · arXiv · Quan Zhang et al. · Mar 30, 2026
- Corvic's own tests find its agent vault keeps keys out of prompts but does much less against a misused connector
Paper · · arXiv · Kenney, Corvic AI · Sep 27, 2026
- The IBAC paper's strict mode blocked every prompt injection, and its agent finished only a third of its tasks
Article · · ibac.dev · Jordan Potti · Mar 3, 2026
- In one author's MCP benchmark, a confirmation prompt on every call still let 6 of 10 attacks through
Paper · · arXiv · Ting Liu, SymbolicLight · Jun 27, 2026
- AuthGraph checks where an agent's tool inputs come from, but trusts a poisoned source the user pointed it to
Paper · · arXiv · Peiran Wang et al. · May 26, 2026
- A forged invoice gets past an auditor that traces which text drove each AI agent action
Paper · · arXiv · Shihao Weng et al. · May 5, 2026
- Checking each agent call against the user's own task stopped all 634 off-task calls the authors wrote
Paper · · arXiv · Reshabh K Sharma et al. · Mar 17, 2026
- A third-party agent talked a trusted one into unlocking the door; SEAgent blocks it, given the right labels
Paper · · arXiv · Zimo Ji et al. · Jan 17, 2026
- AgentGuardian's learned rules caught 18 of 20 misuse cases; its sample rule, as printed, allows any text file
Paper · · arXiv · Nadya Abaev et al. · Jan 15, 2026
- Attacks tuned to each defense got past all 12 prompt-injection and jailbreak defenses tested
Paper · · arXiv · Milad Nasr et al. · Oct 10, 2025
- Meta's injection-trained Llama kept agent attacks to 2.1% or less, but its authors tried no attack tuned against it
Paper · · arXiv · Sizhe Chen et al. · Jul 3, 2025
- RAS-Eval cites weak MCP authentication, yet maps none of its 3,802 agent attacks to an authentication weakness
Paper · · arXiv · Yuchuan Fu et al. · Jun 18, 2025
- In Agent Security Bench, eight text-level defenses left agent hijacking high; none checks the tool call
Paper · · arXiv · Hanrong Zhang et al. · May 30, 2025
- In Microsoft's agent tests, label checks blocked policy-breaking tool calls and its FIDES planner finished more tasks
Paper · · arXiv · Manuel Costa et al. · May 29, 2025
- Meta's LlamaFirewall cut agent attacks to 1.75% in a replay that counts late catches as defended
Paper · · arXiv · Sahana Chennabasappa et al. · May 6, 2025
- Most of Progent's protection from prompt injection comes from a policy written before the agent reads anything
Paper · · arXiv · Tianneng Shi et al. · Apr 16, 2025
- Google's CaMeL stops nearly all prompt injection in tests, but agents finish fewer tasks and need written policy
Paper · · arXiv · Debenedetti · Mar 24, 2025
- Splitting an agent's privileges stopped every tested prompt injection, if users refuse each alert
Paper · · arXiv · Juhee Kim et al. · Mar 17, 2025
- A fake error message relayed between agents got three multi-agent frameworks to run an attacker's code
Paper · · arXiv · Harold Triedman et al. · Mar 15, 2025
- All eight prompt-injection defenses in this test were bypassed by attacks tuned to each one
Paper · · arXiv · Qiusi Zhan et al. · Feb 27, 2025
- A Carnegie Mellon defense blocked every policy-breaking injection in tests, if its labels and tools hold up
Paper · · arXiv · Peter Yong Zhong et al. · Feb 13, 2025
- NIST's best new attack hijacked a Claude agent on 81% of test tasks, up from 11%; retries helped attackers too
Article · · NIST Center for AI Standards and Innovation · CAISI technical staff · Jan 17, 2025
- AgentDojo's best defence against prompt injection was a tool list fixed before the agent reads anything
Paper · · arXiv (NeurIPS 2024 Datasets and Benchmarks) · Edoardo Debenedetti et al. · Jun 19, 2024
- The 2023 paper that named indirect prompt injection showed an attacker can use whatever access the assistant has
Paper · · arXiv (cs.CR) · Kai Greshake et al. · Feb 23, 2023
Related terms
What appears alongside AgentDojo in the same briefings.
RAS-EvalCaMeLInjecAgentGPT-4Agent Security BenchProgentFIDESIsolateGPT