Claude Opus 5 Is the Hardest Model to Inject Yet. That Is Not the Same as Safe.

The claim going around since the Claude Opus 5 release is that it is unusually hard to prompt-inject. It is worth saying plainly: the claim is correct, and the receipts are in section 5.2 of the system card. On the Gray Swan Indirect Prompt Injection benchmark, the probability that an attacker finds a working attack within fifteen attempts fell from 5.5% against Opus 4.8 to 2.0% against Opus 5. At a single attempt it fell from 0.5% to 0.2%. Anthropic calls it the most robust model it evaluated, and against the field that holds up: the best non-Claude model on the benchmark was Muse Spark at 16.5% within fifteen attempts — more than eight times Opus 5's rate — and the most capable GPT 5.6 variant, Sol, came in at 20.0%, ten times Opus 5. One line is worth reading twice: a single attempt against GPT 5.6 Sol succeeded 3.1% of the time, higher than what an attacker achieved against Opus 5 after fifteen.

That is a real result on a hard problem, and it deserves to be reported as one. It is also, read carefully, a more interesting document than the headline number suggests.

What the benchmark actually measures

Prompt injection is a malicious instruction hidden in something the agent reads while doing its job — an email it was asked to summarize, a webpage it was told to browse, a document in a shared drive. The agent processes the tool result and follows the buried instruction as if the user had typed it. The reason it is the hardest problem in agent security is the combination Anthropic names directly: it is dangerous specifically when a model can both access private data and take actions on the user's behalf, because that pairing is what turns a bad instruction into exfiltration or an unauthorized transaction. And it scales — one payload on a public page compromises every agent that reads it, with no need to target anyone in particular.

The IPI benchmark replaced Anthropic's older Agent Red Teaming benchmark, which had been retired because Claude models were at or near its ceiling and it no longer distinguished between releases. IPI was built with Gray Swan, the UK AI Security Institute, the US Center for AI Standards and Innovation, and other model developers: 28 scenarios drawn from a public red-teaming competition, aimed at inducing irreversible harm — data exfiltration, data destruction, system compromise, unauthorized financial transactions — and 1,130 attacks selected after deduplication for high transferability across targets.

Note the asymmetry in how the scores were produced. Claude models were evaluated without additional safeguards. Other frontier models were evaluated on their publicly available endpoints, "which may or may not include additional safeguards." The comparison is, if anything, unflattering to Claude by construction, which makes the gap more credible rather than less.

The adaptive numbers, surface by surface

Anthropic is blunt about the limits of the headline benchmark: "Fixed datasets of known attacks can provide a false sense of security, as a model may perform well against established attack patterns while remaining vulnerable to novel approaches." So the more informative results are the adaptive ones, where an attacker refines its attacks against the model over hundreds of attempts.

In coding environments, tested with Gray Swan's Shade adaptive red-teaming tool across 40 scenarios at 200 attempts each, Opus 5 was hit on 0.56% of attempts with extended thinking and 0.41% without. Opus 4.8's numbers on the same test were 7.03% and 17.44%. That is not an incremental gain; the non-thinking configuration improved by a factor of forty.

In computer use — the model driving a GUI directly — Opus 5 dropped from Opus 4.8's 7.14% to 0.54% with thinking, and from 6.21% to 0.39% without. Every Opus 5 configuration broke exactly one of the fourteen scenarios.

In browser use, evaluated across 129 curated environments in the Claude Cowork harness, Opus 5 was injected on 3.70% of attempts with thinking versus Opus 4.8's 31.5%. With Anthropic's "auto mode" safeguards enabled, no attack succeeded against Opus 5 in either configuration: 0 out of 129 scenarios.

Three things the same pages tell you

First: the zeros belong to the system, not the model. The 0% browser-use figure is the auto mode row. Auto mode is two independent layers — injection probes that inspect tool results before the model acts on them, and a classifier that blocks dangerous tool calls before they execute. Anthropic's own framing of why it works is the important sentence in the whole section: the probes act on data coming in and the classifier on actions going out, "so an attack has to defeat both independently to succeed." The raw model, on that same browser evaluation, sits at 3.70%. The distance between 3.70% and 0% is architecture, not weights. Anthropic notes that Claude Cowork never actually runs without safeguards; the unprotected column exists only to isolate model behavior from product behavior.

Second: Opus 5 is not uniformly the most robust Claude. It leads on IPI. It does not lead everywhere. On the adaptive coding evaluation, Sonnet 5 was more robust (0.31% with thinking, versus Opus 5's 0.56%), as was Mythos 5 at 0.45%. On unprotected browser use, Sonnet 5 was better by a wide margin — 0.93% against Opus 5's 3.70%. Anthropic reports all of this in plain sight. "Most robust model" is a claim scoped to a benchmark, and the frontier model is not automatically the safest choice for a given surface.

Third: the live-adversary number is not in yet. A footnote records that the bug bounty in which outside participants attack the newest model directly "is still being set up; we will update this section once results are available." Everything above is Anthropic and its research partners attacking Anthropic's model. That is rigorous work, done more honestly than most, and it is still not the same thing as the internet trying.

The residual is the number that matters

Anthropic states the threat model it used and states that it is deliberately permissive: the adaptive attacker optimizes directly against the test scenarios and gets many attempts per scenario, affordances a real attacker usually lacks because the target deployment is unknown and repeated attempts raise the odds of detection. That cuts in the defender's favor, and it is a fair point.

But there is a mirror-image caveat that the card cannot address, because nobody at a model lab can: those scenarios are not your deployment. The benchmark's 28 scenarios and the adaptive suite's 40 coding cases were built to resemble real tasks in general. Your agent has a specific system prompt, a specific tool inventory, a specific retrieval corpus, and a specific set of things it is permitted to do with money, data, and other systems — none of which any of these evaluations saw. A model that resists a transferable attack 98% of the time is a much better starting point than one that resists it 80% of the time. It is still a starting point.

So take the good news at full strength: model-level injection robustness improved by roughly an order of magnitude in one release, the layered product safeguards drove a real benchmark to zero, and the spread between vendors is now wide enough that model selection is itself a security control worth documenting. That is genuine progress, and picking the more robust model is one of the cheapest risk reductions available right now.

Then notice what did not happen. The residual did not reach zero. It got small, measurable, and — critically — dependent on choices that belong to you rather than to Anthropic: which model, which surface, which safeguard configuration, which permissions, which blast radius when the 2% lands. A 2.0% chance that an attacker finds a working attack is a very different exposure on an agent that can read a calendar than on one that can move money.

That gap is the whole job. Guardrails narrow the residual and the last two system cards show they are narrowing it fast. What remains after the guardrails is not a rounding error to be assumed away — it is a quantity to be measured against your actual deployment, reduced where it can be, and priced and carried where it cannot. Measuring how AI agents fail and insuring what is left is the work Auly exists to do, and the better these numbers get, the more precisely that remainder can be named.