Jailbreak
jailbreaking · universal jailbreak · non-universal jailbreak
A prompt or technique that gets an AI model to bypass its safety safeguards; a 'universal' jailbreak works broadly across topics, a narrow one only in specific circumstances.
In our briefings
The claim each of the newest briefings on Jailbreak would have you leave with.
Mattson, SecureAuth's chief executive, argues that a model cannot be secured the way tested software is: it is non-deterministic, and a request phrased another way, or in another language, gets around whatever it was told not to do. His advice is to "just assume that it could do anything any time" and put the guardrails around the model, so that a financial transaction or a database change the agent attempts is approved and recorded. He cites Anthropic as saying no guardrail is adequate. The statement we take him to mean, Anthropic's of 12 June, which he recalls as "three weeks ago, I believe", says the company suspects perfect jailbreak resistance "is not currently possible for any model provider", while defending its own safeguards as strong. Briefings we have published on other sources move the control off the model too: a Silverfort guest calls guardrails on a probabilistic system guidance rather than control, ClearVector's John Laliberte compares them to bowling bumpers, and a zero-trust framework paper for agents adds a session-enforcement layer.
When the model cannot be trusted, control what the agent can reach, checked on every actionBI-FIND-2026-0151
LeVi's case is that a code model is about as predictable in its bugs as in the numbers it picks: "there's no creativity in number, there's no creativity in code generation, also not in vulnerabilities." So apps built by the same model tend to carry the same flaws. His lab's FSTab method is built on that: once it knows which model wrote an app, it takes features anyone can see, such as login, file upload or password reset, and predicts which backend flaws the app probably has. A demo during the talk surfaced a cross-site scripting flaw. The paper behind the talk states its limits: every test app was one the researchers generated, the flaws it counts are code-scanner findings that it calls predictions rather than guaranteed exploits, and the method assumes the attacker knows or can infer which model wrote the app. A manual audit by the authors found 82% of the CodeQL findings and 74% of the Semgrep findings to be genuine insecure patterns.
Apps built by the same code model repeat its security flaws, and their visible features hint at which onesBI-FIND-2026-2041
Revision -03 adds a rule -02 did not have: the token exchange belongs to the infrastructure layer that runs the model's tool calls, and the model must not be able to call it or bypass it. Section 3.7 says the exchange must not be exposed as a tool the LLM can invoke, the model's output must not feed an exchange request without validation and approval by the infrastructure layer, and "Access tokens, Txn-Tokens, and other credential material MUST NOT appear in the LLM's context window, prompt, or generated output." The reason given is that an LLM can be adversarially influenced, for example through prompt injection or a jailbreak. A harness, a gateway or a sidecar are offered as non-normative ways to comply. Section 5.16 names the limit: a Txn-Token "does not carry in-band evidence" that isolation was enforced, so the draft says deployments should give resource servers that assurance out of band, for example through hardware attestation or signed deployment manifests.
Raut's revised agent draft keeps token exchange out of the model's reach, and concedes the token carries no evidence of itBI-FIND-2026-1960
›Cited in 27 briefings
Every briefing that discusses this term, newest first.
- When the model cannot be trusted, control what the agent can reach, checked on every action
Podcast · · Tech Talks Daily · AI Agent Security: Why Identity and Access Control Matter More Than Guardrails · Aug 11, 2026
- Apps built by the same code model repeat its security flaws, and their visible features hint at which ones
Video · · Black Hat · Uploaded November 3, 2026
- Raut's revised agent draft keeps token exchange out of the model's reach, and concedes the token carries no evidence of it
Draft · · IETF Datatracker · Ashay Raut · May 22, 2026
- Anthropic asks for per-agent identity; its guide leaves Claude Code sub-agent credentials unspecified
Article · · Anthropic (Claude) · Anthropic · May 27, 2026
- Anthropic says agents should never use static API keys, yet its sub-agents can carry the parent's permissions
Article · · Anthropic · Anthropic · May 18, 2026
- Told to block foreign nationals from a model, Anthropic had no way to find them and blocked everyone
Article · · Anthropic · Jun 12, 2026
- The first largely AI-run intrusion campaign got in by telling the model the attackers were a security firm
Article · · Anthropic · Nov 13, 2025
- The directors' own handbook now puts one question about AI agent identity to boards
Article · · National Association of Corporate Directors and Internet Security Alliance · Larry Clinton et al. · Mar 12, 2026
- Refuse the risky action when its plan relies on untrusted input, a Beihang survey of agent attacks proposes
Paper · · arXiv · Guanquan Shi et al. · Dec 7, 2025
- A "don't ask again" for Python let a tampered Agent Skill upload slides from Claude Code
Paper · · arXiv · David Schmotz et al. · Oct 30, 2025
- Attacks tuned to each defense got past all 12 prompt-injection and jailbreak defenses tested
Paper · · arXiv · Milad Nasr et al. · Oct 10, 2025
- A poisoned MCP tool got an agent to copy the client's config file without asking; deleting a file did ask
Paper · · arXiv · Yongjian Guo et al. · Aug 18, 2025
- MCP-Guard's classifier alone came within half a point of its best full pipelines' F1 on the authors' test set, and ran faster
Paper · · arXiv · Wenpeng Xing et al. · Aug 14, 2025
- A Tsinghua blockchain design for AI agents backs its defense claims with a table and reports no attack run
Paper · · arXiv · Zhenhua Zou et al. · Aug 2, 2025
- Meta's injection-trained Llama kept agent attacks to 2.1% or less, but its authors tried no attack tuned against it
Paper · · arXiv · Sizhe Chen et al. · Jul 3, 2025
- A survey of MCP and A2A attacks pairs each identity threat with a fix, and tests none of them
Paper · · arXiv · Mohamed Amine Ferrag et al. · Jun 29, 2025
- Each of eight AI models fell for at least one jailbreak planted in a simulated MCP server
Paper · · arXiv · Junfeng Fang et al. · Jun 16, 2025
- Meta's LlamaFirewall cut agent attacks to 1.75% in a replay that counts late catches as defended
Paper · · arXiv · Sahana Chennabasappa et al. · May 6, 2025
- A fake error message relayed between agents got three multi-agent frameworks to run an attacker's code
Paper · · arXiv · Harold Triedman et al. · Mar 15, 2025
- Testing AI agents one at a time misses attacks that combine them, a Cooperative AI Foundation report argues
Paper · · arXiv · Lewis Hammond · Feb 19, 2025
- A Reddit post and a fake site were enough to make AI agents leak card numbers and phish the user's family
Paper · · arXiv · Ang Li et al. · Feb 12, 2025
- In a lab test with simulated hops, one poisoned email spread through replies that an AI assistant drafted
Paper · · arXiv · Stav Cohen et al. · Jan 30, 2025
- SecAlign held optimised prompt injections to 14% or less on five models; its authors say it cannot stand alone
Paper · · arXiv · Sizhe Chen et al. · Oct 7, 2024
- A 2024 USENIX benchmark found none of ten prompt-injection defenses sufficient, its best detector included
Article · · USENIX Security 2024 · Yupei Liu et al. · Aug 14, 2024
- One hijacking question cut a Gemini Ultra agent's privacy score to 33.1%; choosing its data first kept 93.2%
Paper · · arXiv · Eugene Bagdasarian et al. · May 8, 2024
- OpenAI's instruction hierarchy ranks who may give orders by where the text arrived, not by who sent it
Paper · · arXiv · Eric Wallace et al. · Apr 19, 2024
- StruQ held hand-written prompt injections to 2% or less on two small models; optimised attacks still got through
Paper · · arXiv · Sizhe Chen et al. · Feb 9, 2024