AI R&D capabilities in public models are improving quickly and getting harder to measure, according to METR’s Q4 2026 capability update. The evaluation group tested public models up to Opus 5.5 and GPT-6 Astra and concluded that current public evidence can’t rule out drastic AI R&D acceleration in the next couple of years.
METR defines that scenario as one where capabilities improve so fast that humans can no longer meaningfully understand, assess and react to catastrophic risks from new systems. It also says it remains significantly uncertain about what capabilities that would actually require, and how fast the relevant ones are rising.
Why the old yardstick is wearing out
METR introduced its time horizon metric in March 2025, when the best model, Sonnet 3.7, had a measured horizon of around one hour. Within a year, models saturated the TH1.1 suite, which includes tasks up to 30 hours. In many domains, METR now finds it hard and expensive to recruit human baseliners who beat the models.
The group also says public frontier models are strong enough at coding and hacking that incapability on verifiable software tasks no longer supports an argument that risk is very low. That pushed it to add a harder task set, TH2, and to score each task against multiple thresholds.

What the results show
On METR’s coding and optimization tasks, models usually match or beat skilled humans. When it recruited its own baseliners, it typically couldn’t find anyone who outperformed the frontier models. Where it had ten or more domain specialists attempting a task, models landed at a very high percentile but not at the very top.
Several other findings stand out:
- Time horizons: on the combined TH1.1 and TH2 suite, horizons are doubling roughly every 3.6 months. That is similar to a trend OpenAI recently reported on internal research tasks.
- Possible acceleration: the doubling time is shorter on a verifiable subset of TH2 when TH1.1 is excluded. That could mean a recent speedup is hidden in the combined trend, though METR notes other explanations are plausible.
- Cost of realism: tasks that are costlier to turn into reinforcement-learning environments are somewhat harder for models. A doubling of a task’s lifecycle cost carries a penalty very roughly equal to halving the agent’s time horizon. Yet progress rates look similar on cheap and costly tasks.
- Judgment: on tasks that need choices without clear empirical feedback, AI agents show worse judgment than human experts. In one task, model judges overestimated the quality of outputs made by similarly strong or stronger models, compared with an expert scorer.

A composite index shows no clear break
METR also tracks public frontier capability with the METR Capability Index, built using item response theory, a statistical method similar to the one behind the Epoch Capabilities Index. Across the public models it tested, it saw no strong acceleration or deceleration in the overall trend. It did see uncertain evidence of developer-specific slope changes under some conditions, but it isn’t sure they’re valid because the results shift with methodological choices.

Checks on the numbers
METR ran sanity checks, including for reward hacking and refusals. It also tested agent swarms as a way to draw out stronger performance. Swarms looked helpful on verifiable tasks with a clear metric and fast feedback, compared with best-of-k sampling, and had unclear or negative effects on the others.
What measurement needs next
Tasks with clear scoring are close to saturated. The skills that could matter most for runaway progress, such as judgment under weak feedback, are harder to test and cost more to build. METR’s discussion section lists open questions on how to assess acceleration, including which skills are actually necessary and how quickly they are improving.
Anyone using a single benchmark curve to gauge risk from AI R&D capabilities should keep that in mind. The group’s own position is that the evidence is too thin to rule out a fast jump, and that better tests of AI R&D capabilities are needed before anyone can say otherwise.
Source: metr.org