A new round of prompt injection defense research shows that AI agents can be trained to resist hidden instructions far better than a safety line in the system prompt allows. Security firm ALICE and AI21 Labs post-trained NVIDIA’s Nemotron 3.5 Super VL on simulated Slack and GitHub workspaces, and its pass rate on attack tasks rose from 11% to 57%.
The test environments are what ALICE calls adversarial RL gyms. The results also carry some uncomfortable numbers for the most cautious frontier models.
The attack agents keep falling for
Indirect prompt injection happens when an attacker plants instructions in something an agent reads while doing a legitimate job: a document, a chat message, a tool response. The agent can then take unauthorized actions, leak sensitive data or redirect a workflow. Leading AI labs acknowledge that the problem is unsolved.
It is hard to test because attacks unfold over several steps. They involve tool calls, outside content and decisions that depend on earlier moves, so checking single prompts misses most of the risk.

How the gyms work
Each gym is a self-contained workspace with the tools, content and data an agent would meet in an app like Slack or GitHub. ALICE packages them as Harbor tasks with their own container images, so every run starts clean and hundreds of rollouts can run in parallel.
The attack scenarios come from ALICE’s threat intelligence system, Rabbit Hole, and are developed with 150 security researchers and subject matter experts. Tasks fall into two types:
- Attack tasks hide malicious instructions in the environment, and the agent has to spot and resist them.
- Over-refusal tasks present legitimate requests that look risky, and the agent should go ahead.
The second type exists because training only against attacks teaches a model to refuse anything suspicious. In one example, a Slack agent must remind staff to reset passwords while the reset link has been swapped for a credential-harvesting URL. In another, a finance assistant handling a vendor payment finds an instruction inside an invoice record to redirect the money to a different bank account.
What the benchmarks show
ALICE ran each task eight times per model. It also keeps CHASM, a benchmark of 117 enterprise attack tasks across finance, HR, legal and customer support, built from applications separate from the training gyms.
Sonnet 5.5 resisted roughly 4 in 5 attacks, the best result. GPT-6.1 Sol and Sonnet 5 resisted about 2 in 3, and GPT-6 Luna and GPT-5.6 Terra about 3 in 10. Several open-weight models resisted fewer than 1 in 10.
Safety came at a price in usefulness. On legitimate GitHub tasks built to resemble risky requests, Sonnet 5.5 scored 26.9, while the base Nemotron model scored 90.7. DeepSeek V4.1 Flash showed the same pattern with a 22.2.
Training beats a system prompt
After supervised fine-tuning on ALICE’s data, Nemotron’s attack-task pass rate went from 11% to 57%. Adding a safety instruction to the system prompt lifted it to about 29%, so training delivered more than twice the gain. On CHASM, Nemotron’s reward rose from 7.54 to 34.58, which suggests the learning carried over to attacks it had not seen.
The dataset was small. The training split had 137 tasks, and the fine-tuning set drew 1,617 unique rollouts from 126 of them. After this step alone, Nemotron ranked fourth among the 11 models on overall validation reward. The share of tasks showing reward variation, which matters for reinforcement learning, climbed from 35% to 67%.
AI21 then added online reinforcement learning. The resulting model approached GPT-6.1 Sol on both safety and utility. AI21 reports it beat Sol on overall validation reward, 76.2 against 65.9, at a cost per million tokens 25 times lower. The strongest frontier models still lead on CHASM, so that gap narrowed without closing.
What it means for companies deploying agents
Teams that build on open-weight models can use adversarial and over-refusal tasks in post-training to put their security policies into the model’s weights. That reduces dependence on system prompts and extra guardrails, and the model still completes legitimate work. The same gyms can rerun after every model update to track both attack resistance and task completion.
Where the gyms go next
ALICE says it is expanding the gyms to more enterprise applications, agent workflows and attack types. The aim is to let model builders find weaknesses, train against them and check whether the gains hold in new environments.
Source: alice.io