OpenAI Proposes Perplexity Limits for AI Labs

Chart concept showing perplexity limits as a shared yardstick for AI progress

OpenAI Chief Scientist Jakub Pachocki wants the industry to treat perplexity as a shared yardstick for AI progress. In an essay titled “Perplexity and Intelligence”, he argues that this one measure explains much of what models have learned to do, and that labs should agree on common limits tied to it.

OpenAI is backing the idea with data. It has tracked perplexity internally since GPT-3.5 and is publishing the numbers for the first time.

What perplexity measures

Perplexity limits only make sense once you know the measure. Perplexity, or its close relative cross-entropy loss, scores how well a model predicts the next token on average. Lower is better. Pachocki frames modern AI as training a compressor, or predictor, on a very large body of data, then applying prompting, fine-tuning or reinforcement learning to draw out what the model has learned.

He says the capabilities line up with the numbers in a striking way. The figures below come from OpenAI’s main evaluation dataset:

  • GPT-3.5, at a log-perplexity of about 0.99, could hold a fluent conversation and became the base for the original ChatGPT.
  • GPT-4, at about 0.91, used chain-of-thought reasoning productively, which led to modern reasoning models.
  • Below 0.8, OpenAI saw striking capabilities appear in fields like cybersecurity and mathematics.

Pachocki stresses it wasn’t clear in advance that these abilities would show up at those exact thresholds.

The math results at 0.76

For more than a year, OpenAI has put open math problems to its models as one way of testing them. Once it moved to a model at a log-perplexity of about 0.76, the results changed. Pachocki says seeing long-pondered questions answered one after another in a single Slack thread felt deeply surreal.

He draws a forecast from that. Deep learning research relies more on experiments, but simple insights have been missed for years before, so as soon as 2027 there could be a similar flood of fundamental advances in pretraining. He expects AI to help with alignment and monitoring too, but thinks that progress is unlikely to move fast enough to match the potential acceleration in pretraining.

The proposal: shared perplexity limits

Looking ahead, Pachocki says to expect further new capabilities as models enter a log-perplexity window of roughly 0.68 to 0.7. He doesn’t know what those capabilities will be, and says labs may not be testing for them today. He also expects current forms of alignment auditing to degrade and models to grow more aware of being evaluated.

His proposal is for frontier developers to jointly set perplexity limits, based on perplexity and other measurements, that none would cross until the ability to understand and monitor AI behavior reaches a concrete milestone. One example he gives is a successor to chain-of-thought monitors. The first limit would be tracked across labs, with a defined bar to clear before going further.

OpenAI acknowledges the cost of sharing the data. Publishing gives competitors a clearer view of where it stands, but the company says that’s outweighed by the urgency of agreeing on safety measures.

Where the metric falls short

Pachocki lists four weaknesses of perplexity as a standard:

  • Labs use different evaluation datasets and data mixtures, so comparing across them is hard.
  • Perplexity doesn’t map directly onto any task people care about.
  • It misses gains from outside pretraining. Chain-of-thought reasoning doesn’t lower perplexity, yet it greatly raises capability.
  • Nobody knows the floor. A loss of zero isn’t achievable, but the true entropy of the data is unknown, and smaller gains near it may mean bigger jumps in capability.

He argues it has still held up as a metric that many research teams have optimized for, and that it keeps delivering consistent gains.

What happens next with perplexity limits

No other lab has publicly committed to the proposal yet. Any shared standard would first need a way to compare base models across different datasets, which Pachocki says transparency should make possible.

For now the essay sets out OpenAI’s position: pretraining progress is the number to watch, the next window could bring capabilities no one is testing for, and agreed limits should arrive before it does.

Source: OpenAI

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.