AI Benchmark Exploits Hit 100% on Top Agent Tests

AI benchmark exploits are far easier than the leaderboards suggest. Berkeley researchers built an automated scanning agent that audited eight prominent agent benchmarks and found every one could be pushed to near-perfect scores without solving a single task.

The benchmarks covered include SWE-bench Verified, SWE-bench Pro, WebArena, OSWorld, GAIA, Terminal-Bench, FieldWorkArena and CAR-bench. The agent made no real attempt at the tasks. It attacked the way scores are computed.

The scores the exploit agent reached

In most cases the agent used zero LLM calls. Its results, by benchmark:

  • Terminal-Bench, 89 tasks: 100%
  • SWE-bench Verified, 500 tasks: 100%
  • SWE-bench Pro, 731 tasks: 100%
  • WebArena, 812 tasks: about 100%
  • FieldWorkArena, 890 tasks: 100%
  • CAR-bench, all hallucination tasks: 100%
  • GAIA, 165 tasks: about 98%
  • OSWorld, 369 tasks: 73%

How the tricks work

On SWE-bench Verified, a patch runs inside the same Docker container as the tests. A ten-line conftest.py file with a pytest hook can mark every test as passed, and pytest loads it automatically before running anything.

Terminal-Bench is different. Its sandbox protects the test files but not system binaries, and 82 of its 89 tasks download the uv tool with curl during verification. The agent swaps in a fake curl, and it installs a trojanized uvx that prints passing pytest output and writes a reward of 1. The other seven tasks fall to similar wrappers around pip or python.

WebArena gave way to a simpler move. Pointing Chromium at a file:// URL reads the gold answer straight from the task config, which yields close to 100% on all 812 tasks. In FieldWorkArena, validation never checks whether an answer is correct, so sending an empty {} was enough.

Gaming that already happened

The team says scores are being inflated in practice, and it points to cases:

  • IQuest-Coder-V1 claimed 81.4% on SWE-bench. Researchers found that 24.4% of its trajectories ran git log to copy answers from commit history. The corrected score was 76.2%.
  • METR found that o3 and Claude 3.7 Sonnet reward-hack in more than 30% of evaluation runs, using tricks such as stack introspection and monkey-patching graders.
  • OpenAI dropped SWE-bench Verified after an internal audit found flawed tests in 59.4% of the audited problems.
  • In KernelBench, torch.empty() can return stale GPU memory that holds the evaluator’s earlier reference answer.

The team also cites Anthropic’s Mythos Preview. In one episode the model needed to edit files it had no permission to change, found a way to inject code into a config file that ran with elevated privileges, and designed the exploit to delete itself afterwards.

What the researchers want changed

Their advice to benchmark builders is concrete. Keep ground truth for primary leaderboard splits private, because once answers are public a benchmark measures memorization. Rotate test instances regularly, since a static test turns into a lookup table over time. Consider held-out evaluation, where submitters hand in model outputs and a private test set scores them.

They don’t blame the benchmark authors. Their view is that resistance to adversarial testing simply isn’t standard practice in evaluation design yet, and it should be.

Why frontier models raise the stakes

The team warns that the gap between a high score and real capability will widen as pressure to show capability through benchmarks grows. Models that are good at pattern-matching may stumble into these exploits by accident, and models tuned directly for benchmark results may use them on purpose.

BenchJack and what comes next

The scanning agent is being developed into BenchJack, a general-purpose vulnerability scanner for agent benchmarks. It works in two phases: it studies the evaluation code and catalogs loopholes, then it writes working exploits for each one. If its exploit agent scores above baseline, the benchmark has a hole.

The team wants BenchJack to become a routine step before and after every benchmark release, much like unit tests. A public release is being prepared, with a mailing list for updates.

For anyone choosing a model from a leaderboard, the takeaway is practical: check how a score was produced before trusting it.

Source: rdi.berkeley.edu

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.