MedChemBench Tests AI on Drug Candidate Choices

Molecular structures illustrating the MedChemBench benchmark for AI drug discovery

OpenAI has introduced the MedChemBench benchmark, a test of whether AI models can reason like medicinal chemists. It covers 99 problems built from 117 published papers and patents, with 761 distinct molecular structures between them. The best result so far: GPT-6.2 Astra collected 84.5% of possible points at the xhigh reasoning setting.

The benchmark asks models to link a molecule’s structure to experimental evidence, explain results, rank compounds and forecast their properties. It doesn’t claim to show that AI can discover drugs.

Why drug design is a tradeoff problem

Improving one property of a molecule often damages another. A compound might hit a disease-related protein hard but be absorbed poorly, or break down too fast in the body. Chemists handle this by making and trying out families of similar compounds, and each round of results informs the next batch.

For AI to help, a model has to connect small structural changes with measured outcomes and judge which compounds deserve a lab slot.

Introducing MedChemBench — Header poster 4K
Image: OpenAI

How the benchmark is built

In most problems a model is handed a snapshot from the middle of a research effort: what the drug is meant to hit, what the team wants, which compounds exist already and a few lab results. Scoring splits into four parts:

  • Molecular structures: getting the atom connections and 3D orientation right when building molecules from drawings and chemical groups.
  • Scientific reasoning: explanations backed by the supplied evidence, such as why one compound beats another.
  • Ranking and selection: picking compounds that satisfy the brief.
  • Property predictions: closeness to the measured and calculated values reported in the source material.

Predictions earn partial credit based on how close they land to measured values.

Introducing MedChemBench — PRMT5 poster 4K
Image: OpenAI

A PRMT5 example

One problem concerns PRMT5, a protein targeted in cancer research. The model sees structures and earlier results, then forecasts three things for a trio of related compounds: potency against PRMT5 in cells, passage through a cell barrier, and the speed at which liver enzymes degrade them in a lab test.

Follow-up experiments, kept from the model, revealed a tradeoff. Compound 23 needed a lower dose to inhibit PRMT5 in cells. Compound 24, by contrast, passed the barrier better and lasted longer against the liver enzymes.

Molecular reconstruction comparison: GPT-6 Astra preserves the correct 18-atom ring, GPT-5.6 Sol produces a smaller ring, and GPT-5.6 Luna produces disconnected fragments.
Image: OpenAI

The scores

OpenAI tested eleven GPT models at several reasoning settings. At xhigh, the setting all eleven share, GPT-6.2 Astra earned 84.5%, GPT-6 Astra 83.3%, GPT-6.1 Sol 82.0% and GPT-5.6 Sol 58.2%. GPT-6.1 Terra scored 75.1%. Pushed to maximum effort, GPT-6 Astra got to 83.8%. More reasoning effort raised scores, yet wide gaps between models stayed.

Structure work showed the sharpest difference. One task has a model build a molecule from a core and two side groups while preserving an 18-atom ring. GPT-6 Astra got it right in ten of ten tries. GPT-5.6 Sol and GPT-5.6 Luna missed every time: Sol drew a 15-atom ring, and Luna broke the molecule into two separate pieces. Either error changes which molecule is being analyzed.

Across the benchmark, GPT-6 Astra earned 95.4% of the structure credit at xhigh. It was also strong at explaining experimental evidence.

Where models still fall short

Choosing compounds and predicting numerical properties proved harder. For those categories the model earned only about half the available credit. The tasks use different scoring rules, so the figures aren’t directly comparable, but they point to real room for improvement.

The authors add several caveats. Prior exposure to the published sources can’t be ruled out. Each response was a single attempt with no tools, feedback or new experiments. The results therefore don’t show whether models can find new drugs or improve decisions in a working lab.

Why the structure step comes first

Before a model can say anything useful about how a compound behaves, it has to represent the compound correctly. A drawing carries bonds, rings and spatial arrangement, and a model that drops any of that is reasoning about the wrong molecule. That is why the benchmark scores structure reconstruction as its own component.

What comes next for MedChemBench

OpenAI says it expects MedChemBench to support continued progress and plans to share updates as its models advance. Selection and prediction scores are the numbers to watch, since those are the skills closest to the choices chemists make every day.

Source: OpenAI

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.