A security researcher has shown that AI hacking agents can be tricked into running code planted by their targets. In an offline lab, a deceptive binary pushed the share of runs where the agent executed it from under 10% to nearly 100%. The test targeted ARTEX, the open-source agentic penetration testing platform that the FBI warned about this month.
The experiment points to a possible defense against AI hacking agents, though its author calls it far from rigorous science.
The FBI warning
On November 13 the FBI issued a FLASH warning saying threat actors had weaponized ARTEX to automate multiple phases of intrusions at machine speed, from reconnaissance through data exfiltration. The bureau said attackers have used it against real organizations since at least August. Censys counted more than 1,600 exposed ARTEX consoles in early November. The developer pulled the repository in October, but forks and archived copies remain.
ARTEX doesn’t ship with a model. It plugs into whatever LLM the operator connects, and a coordinating agent spawns sub-agents to pursue the operator’s goal. Attackers reportedly favor DeepSeek’s Flash family, which is cheap, capable and shows no observable guardrails when a request is framed as penetration testing.
Two weaknesses to exploit
The researcher noticed that agentic attackers are chaotic, probing everything and leaving a trail of activity. That raised a question: could a target manipulate them by deceiving them?
Two traits made it likely. Reinforcement learning rewards models for finishing assigned tasks, so agents tend to take shortcuts whenever a job gets hard. And refusal is itself a trained behavior. Models used by attackers often have it muted or removed, which also removes the habit of avoiding risky actions.
So an agent will do what the defender wants, as long as that’s the easiest way to finish the task.
How the trap worked
The researcher built a multi-step CAPTCHA-style gate, starting with adding numbers and ending with a simple computer vision problem. While testing versions, something odd turned up: when the agents were handed executables, they sometimes just ran them.
The recipe has two parts. Convince the agent the challenge blocks its goal, and leave it no other way past. A plain binary gets disassembled and its keys extracted, so the researcher obfuscated the file heavily to make it unreadable to the model.
Results in the lab
The lab used local models, mostly GLM 5.3 Flash and DeepSeek 4.1 1017 Flash. The CAPTCHA program sat in front of capture-the-flag challenges taken from GitHub, and ARTEX was told to find and exploit vulnerabilities in the service behind it. Across roughly 100 runs on four models, obfuscation lifted the execution rate from probably under 10% to nearly 100%.
The agents’ own logs show them talking themselves into it. One noted it would check the binary’s strings first, then run it with a timeout because it was “the exercise’s intended step.” Another reasoned that running an untrusted binary could be dangerous but was part of the intent, so it would proceed in a “sandboxed-ish” way. The orchestrator often handed out the task, and sub-agents cited its instructions as justification.
Even a model that hasn’t had its refusals stripped out knows better, the researcher says, and does it anyway.
Not a standard prompt injection
Using an attacker’s agent against it isn’t new. In 2024 researchers proposed Mantis, which plants decoy services and hides prompt injections in their replies, including commands that open a reverse shell back to the defender. The researcher doubts modern models would still mistake tool output for their own reasoning. The new approach never tells the model what to do. It offers a familiar path that leads where the defender wants.
The researcher withheld the full code because the binary is malware-adjacent, giving enough detail for a security expert to reproduce it in an isolated lab. Hacking back is illegal in nearly every country, and agents shouldn’t be pointed at systems you don’t control.
What it means for defenders
A common suggestion is to run a hacking agent against your own infrastructure to find holes first. The researcher is no longer sure that’s wise, since such agents can be deceived. As models improve the behavior will likely change, but today’s attack agents are open to deception, and that gap may let builders create active defenses against the AI-driven attacks now hitting the internet.
Source: a16z Frontier