METR Task Horizon Overstates Real-World AI Ability

Chart comparing the METR task horizon with real-world AI task lengths at OpenAI and Google

The METR task horizon, the chart many people use to argue that AI is racing toward superintelligence, may be badly off. New figures from inside OpenAI and Google suggest it overstates what AI can do on real work by 33 to 41 times, and the distance is growing.

The case comes from an analysis that compares benchmark-based projections with the first detailed real-world task data from the labs themselves.

What the METR task horizon measures

METR, an AI evaluation group, measures how long a task would take a human, then checks whether an AI can complete it at a given success rate, usually 80% or 50%. Plotted on a log scale, the line is straight, which means exponential growth. It rose from a few seconds in early 2020 to a four-hour horizon at 80% success for Claude Mythos Preview in early 2026.

The Epoch Capabilities Index (ECI) works as a general score for AI. Among models in METR’s sample, about five ECI points corresponded to a doubling of the horizon at 80% success. Benjamin Todd of 80,000 Hours used Anthropic’s internal index to estimate that the strongest models can complete tasks of 30 to 40 hours.

The original AI 2027 scenario went further, projecting tasks lasting months by now and more than a year by the end of 2027.

At 80% success, model-matched ECI → METR estimates are 33 times OpenAI’s internal horizon and 41 times Google’s. The original AI 2027 scenario gives ratios of 111 and 114.
Image: rameznaam.com

What OpenAI and Google actually report

In September, OpenAI published month-by-month data on task success inside the company, covering January through July. Google DeepMind released a similar chart with its Gemini Argon launch, and OpenAI posted a longer, updated series in November.

At 80% success, AI models handle real tasks of about nine minutes at Google and an adjusted 16 minutes at OpenAI. The OpenAI figure is higher than the company reports because the analysis corrects for ambiguous or unsolvable tasks and for errors in the tools the AI uses. Those adjustments don’t change the overall conclusions much.

  • ECI-to-METR estimates run 33 times OpenAI’s internal task length and 41 times Google’s at 80% success.
  • The original AI 2027 scenario overshoots real-world task lengths by more than 100 times.
  • At 50% success, the overshoot is 23 to 25 times.

METR itself expected a gap, since an expert in a field can finish a task much faster than a generally skilled programmer. That gap was estimated at up to 10 times. These numbers are far larger.

The September 1 ECI → METR frontier reconstruction gives 26.7 hours, 133 times OpenAI’s central estimate or 177 times Google’s.
Image: rameznaam.com

The gap is widening

Exponential growth can absorb a big discrepancy. A 100-fold difference is only about seven doublings, which could close in two years. The trouble is that the discrepancy isn’t holding still.

Between June and October, the divergence between Todd’s chart and OpenAI’s real-world observations at 80% success roughly tripled. Annualized, that would be a 30-fold increase. The gap between the original AI 2027 forecast and reality grew from 37 times to over 100 times, implying growth of more than 25 times a year.

The AI 2027 authors have since revised their model. Their August 2026 update shows a milder version of the same pattern, with the central gap growing about 1.2 to 1.3 times from June to October, roughly a doubling per year.

METR task horizon image 3
Image: rameznaam.com

Is real-world progress still exponential?

Earlier OpenAI data looked less than exponential. The longer series points the other way. At 50% success, the best fit is exponential, with task length doubling every 3.9 months on the central estimate, close to METR’s latest figure. At 80% success, the doubling time is 5.6 months, about 40% slower.

The evidence there is thin. A straight line fits almost as well, and exponential is favored by just 1.36 to 1. Exponential growth is the working assumption, not a settled fact.

The reliability gap

Models improve faster at 50% reliability than at 80%. That suggests the distance between benchmark and reality is even wider at higher reliability. Autonomous self-improvement would likely need success rates well above 90%, and chaining many tasks together could demand more than 99%. Task lengths at those levels can’t yet be measured, so the estimates come from fitted curves and should be treated as extrapolations.

A call for transparency

The analysis uses little Anthropic data because the company hasn’t released the detail OpenAI has, and Google is starting to match. It says METR found Anthropic’s models are accelerating AI progress, but that Anthropic made METR redact the most compelling evidence. The author calls that irresponsible for a company predicting AI will soon double its own pace of progress.

What to watch next

METR has a new report out today and has been working to close the benchmark gap. Watch for whether Anthropic publishes internal task data and whether the next OpenAI and Google updates keep widening the distance from the METR task horizon.

The author’s own advice is simple: forecasters should update as real data arrives, and the field should move from theory to empiricism.

Source: rameznaam.com

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.