Skip to content
IAM Insights

Explainers · Glossary

AgentDojo

A benchmark environment for testing whether a large language model (LLM) agent completes its task while resisting prompt-injection attacks in tool outputs and external content.

In our briefings

The claim each of the newest briefings on AgentDojo would have you leave with.

  1. A delegated mission's authority must be a subset of the parent's, McGuinness argues, and OAuth token exchange (RFC 8693) does not guarantee it. Token exchange carries the delegation, but he calls enforcing the narrowing implementation-dependent and not a protocol guarantee. He writes that in production token exchange deployments today, child tokens routinely inherit the parent's full scope unless the implementation explicitly narrows them, so agents compound a failure that OAuth delegation already has. A child agent may also run in a different runtime or expose weaker telemetry, so authority that is narrower on paper can be less governable in practice. He proposes that the mission's state owner rate each hop's trust quality, meaning how well that hop can be observed and checked, and give a low-rated child a more conservative budget. For a sub-agent an orchestrator spawns at runtime, he says the state owner must still issue its authority, but asks what triggers that issuance. Our briefing on Uber's production design found a token chain that records each agent hop without describing any narrowing. Our briefing on Highflame's open-source ZeroID found narrowing by scope intersection offered as an explicit step.

    OAuth token exchange does not guarantee a sub-agent stays inside its parent's accessBI-FIND-2026-0228

  2. OWASP's AIVSS v0.8 starts from a CVSS v4.0 base score and adds an uplift worth part of the gap between that base and 10. Ten agentic factors, each scored 0, 0.5 or 1, set the share, and an exploit-maturity multiplier scales it, by default 0.97. With the default mitigation factor of 1.0 (no or weak mitigation), all ten worked examples in the document land between 7.1 and 10.0, seven Critical and three High, and a goal-manipulation flaw with a CVSS base of 2.1 comes out at 7.1. The authors read a jump like that as a sign that the agentic deployment context, not the technical flaw, is the main risk driver. On our reading of the formula, a score can only fall below its CVSS base through the mitigation factor, and the document warns that AIVSS scores are not directly comparable to CVSS base scores.

    OWASP's AIVSS lifts a CVSS 2.1 agent flaw to 7.1, and all ten worked examples land High or CriticalBI-FIND-2026-2299

  3. On AgentDyn, the authors' benchmark of 60 open-ended tasks that make the agent replan as it goes, none of the nine defenses they ran on GPT-4o both kept the agent working and stopped attacks. Prompt Sandwiching, Spotlighting and the PromptGuard2 detector finished 55% to 63% of tasks with no attack, at or above the undefended agent's 53.3%, but left 27% to 31% of attacks succeeding. PromptGuard2 kept that work only when no attack came: under attack it finished 20.80% of tasks, because it discards a tool's whole output once it detects an injection. The ProtectAI and PIGuard detectors and the Tool Filter, CaMeL and Progent system defenses held attack success to 4.2% or less and finished 10% of tasks or fewer. DRIFT, built by three of the authors, finished the most tasks of that group with no attack, 30.0%, with 0.8% attack success, and CaMeL finished no task on any of the 12 models tested. The authors' own verdict is that none of them attain "acceptable performance for real-world deployment on AgentDyn".

    None of nine prompt-injection defenses both stopped attacks and kept a GPT-4o agent working on tasks that change mid-runBI-FIND-2026-1854

›Cited in 30 briefings

Every briefing that discusses this term, newest first.

  1. OAuth token exchange does not guarantee a sub-agent stays inside its parent's access

    Article · · Control Plane by Karl McGuinness · Karl McGuinness · Mar 18, 2026

  2. OWASP's AIVSS lifts a CVSS 2.1 agent flaw to 7.1, and all ten worked examples land High or Critical

    Article · · OWASP AIVSS project · Ken Huang et al. · Oct 22, 2026

  3. None of nine prompt-injection defenses both stopped attacks and kept a GPT-4o agent working on tasks that change mid-run

    Paper · · arXiv · Hao Li et al. · Feb 3, 2026

  4. Each agent and each tool call can pass its check while the system fails, an Oxford-led survey argues

    Paper · · arXiv · Christian Schroeder de Witt · Apr 29, 2026

  5. AgentArmor cut attack success to 3% in its authors' tests; most misses came when a user said to follow a file

    Paper · · arXiv · Peiran Wang et al. · Nov 18, 2025

  6. A fixed tool plan blocks prompt injection but breaks tasks; in DRIFT's tests, letting a model widen it wins most back

    Paper · · arXiv · Hao Li et al. · Mar 26, 2026

  7. With real MCP tools for cloud, database and email, most injected attacks got past four models, even with a plan

    Paper · · arXiv · Quan Zhang et al. · Mar 30, 2026

  8. Corvic's own tests find its agent vault keeps keys out of prompts but does much less against a misused connector

    Paper · · arXiv · Kenney, Corvic AI · Sep 27, 2026

  9. The IBAC paper's strict mode blocked every prompt injection, and its agent finished only a third of its tasks

    Article · · ibac.dev · Jordan Potti · Mar 3, 2026

  10. In one author's MCP benchmark, a confirmation prompt on every call still let 6 of 10 attacks through

    Paper · · arXiv · Ting Liu, SymbolicLight · Jun 27, 2026

  11. AuthGraph checks where an agent's tool inputs come from, but trusts a poisoned source the user pointed it to

    Paper · · arXiv · Peiran Wang et al. · May 26, 2026

  12. A forged invoice gets past an auditor that traces which text drove each AI agent action

    Paper · · arXiv · Shihao Weng et al. · May 5, 2026

  13. Checking each agent call against the user's own task stopped all 634 off-task calls the authors wrote

    Paper · · arXiv · Reshabh K Sharma et al. · Mar 17, 2026

  14. A third-party agent talked a trusted one into unlocking the door; SEAgent blocks it, given the right labels

    Paper · · arXiv · Zimo Ji et al. · Jan 17, 2026

  15. AgentGuardian's learned rules caught 18 of 20 misuse cases; its sample rule, as printed, allows any text file

    Paper · · arXiv · Nadya Abaev et al. · Jan 15, 2026

  16. Attacks tuned to each defense got past all 12 prompt-injection and jailbreak defenses tested

    Paper · · arXiv · Milad Nasr et al. · Oct 10, 2025

  17. Meta's injection-trained Llama kept agent attacks to 2.1% or less, but its authors tried no attack tuned against it

    Paper · · arXiv · Sizhe Chen et al. · Jul 3, 2025

  18. RAS-Eval cites weak MCP authentication, yet maps none of its 3,802 agent attacks to an authentication weakness

    Paper · · arXiv · Yuchuan Fu et al. · Jun 18, 2025

  19. In Agent Security Bench, eight text-level defenses left agent hijacking high; none checks the tool call

    Paper · · arXiv · Hanrong Zhang et al. · May 30, 2025

  20. In Microsoft's agent tests, label checks blocked policy-breaking tool calls and its FIDES planner finished more tasks

    Paper · · arXiv · Manuel Costa et al. · May 29, 2025

  21. Meta's LlamaFirewall cut agent attacks to 1.75% in a replay that counts late catches as defended

    Paper · · arXiv · Sahana Chennabasappa et al. · May 6, 2025

  22. Most of Progent's protection from prompt injection comes from a policy written before the agent reads anything

    Paper · · arXiv · Tianneng Shi et al. · Apr 16, 2025

  23. Google's CaMeL stops nearly all prompt injection in tests, but agents finish fewer tasks and need written policy

    Paper · · arXiv · Debenedetti · Mar 24, 2025

  24. Splitting an agent's privileges stopped every tested prompt injection, if users refuse each alert

    Paper · · arXiv · Juhee Kim et al. · Mar 17, 2025

  25. A fake error message relayed between agents got three multi-agent frameworks to run an attacker's code

    Paper · · arXiv · Harold Triedman et al. · Mar 15, 2025

  26. All eight prompt-injection defenses in this test were bypassed by attacks tuned to each one

    Paper · · arXiv · Qiusi Zhan et al. · Feb 27, 2025

  27. A Carnegie Mellon defense blocked every policy-breaking injection in tests, if its labels and tools hold up

    Paper · · arXiv · Peter Yong Zhong et al. · Feb 13, 2025

  28. NIST's best new attack hijacked a Claude agent on 81% of test tasks, up from 11%; retries helped attackers too

    Article · · NIST Center for AI Standards and Innovation · CAISI technical staff · Jan 17, 2025

  29. AgentDojo's best defence against prompt injection was a tool list fixed before the agent reads anything

    Paper · · arXiv (NeurIPS 2024 Datasets and Benchmarks) · Edoardo Debenedetti et al. · Jun 19, 2024

  30. The 2023 paper that named indirect prompt injection showed an attacker can use whatever access the assistant has

    Paper · · arXiv (cs.CR) · Kai Greshake et al. · Feb 23, 2023

Related terms

What appears alongside AgentDojo in the same briefings.

RAS-EvalCaMeLInjecAgentGPT-4Agent Security BenchProgentFIDESIsolateGPT