# OpenAI Releases MedChemBench to Test AI on Real Drug Discovery Tasks

> Source: <https://epyhia.com/openai-releases-medchembench-to-test-ai-on-real-drug-discovery-tasks>
> Published: 2026-11-21 04:05:08+00:00

**Recommendation:** Report per-component scores, not just totals: on MedChemBench structure and rationale carry ~73% of credit, hiding ~50% accuracy on ranking and numerical prediction.
More in the playbook "Evals and testing for LLM apps": <https://wpnews.pro/playbooks/evals-testing>

# OpenAI Releases MedChemBench to Test AI on Real Drug Discovery Tasks

OpenAI published MedChemBench on November 20, a 99-problem benchmark that evaluates whether language models can reason through the kinds of tradeoffs medicinal chemists face when choosing which compounds to make and test next. The benchmark draws on 117 published papers and patents, encompassing 761 distinct molecular structures, and stops each problem at a real decision point in a historical research program — withholding the later experimental results that serve as the answer key [\[1\]](https://openai.com/index/introducing-medchembench)[\[7\]](https://cdn.openai.com/pdf/09c4ec60-a32e-4652-837b-1811368a3bb0/medchembench-paper.pdf).

## What the benchmark measures

MedChemBench groups tasks into seven categories spanning preclinical discovery: integrated lead selection (48 tasks), structure–activity relationship forecasting (26), ADME and delivery (8), safety and liabilities (4), molecular design and enumeration (6), assay biology and mechanism (3), and retrosynthesis (4). Eighty-five of the 99 tasks span more than one scientific domain, reflecting how a lead choice can depend simultaneously on potency, selectivity, cell assays, metabolism, exposure and safety [\[2\]](https://www.unite.ai/openai-introduces-medchembench-for-medicinal-chemistry-reasoning)[\[7\]](https://cdn.openai.com/pdf/09c4ec60-a32e-4652-837b-1811368a3bb0/medchembench-paper.pdf).

Each problem is graded on four components with different weightings: molecular structure reconstruction (37.49% of available credit), scientific rationale (35.78%), ranking and candidate selection (17.51%), and numerical or categorical property prediction (8.92%) [\[7\]](https://cdn.openai.com/pdf/09c4ec60-a32e-4652-837b-1811368a3bb0/medchembench-paper.pdf). Structures are checked deterministically via RDKit canonicalization; scientific rationale is scored by a separate GPT-6 Astra judge at medium reasoning effort; numerical predictions receive partial credit based on fold error relative to reported measurements [\[7\]](https://cdn.openai.com/pdf/09c4ec60-a32e-4652-837b-1811368a3bb0/medchembench-paper.pdf).

## Results show a split between reconstruction and judgment

At the shared xhigh reasoning setting, OpenAI reports GPT-6.2 Astra earned 84.5% of available credit, GPT-6 Astra 83.3%, GPT-6.1 Sol 82.0%, and GPT-5.6 Sol 58.2% [\[1\]](https://openai.com/index/introducing-medchembench)[\[2\]](https://www.unite.ai/openai-introduces-medchembench-for-medicinal-chemistry-reasoning). The technical paper, which evaluated eleven GPT models across 61 model–effort conditions (60,390 responses), records GPT-6 Astra reaching 83.8% at max effort [\[7\]](https://cdn.openai.com/pdf/09c4ec60-a32e-4652-837b-1811368a3bb0/medchembench-paper.pdf).

Component scores for GPT-6 Astra at xhigh reveal where the difficulty lies: 95.37% on molecular structure, 91.05% on scientific rationale, but only 50.42% on ranking and selection and 47.02% on numerical prediction [\[2\]](https://www.unite.ai/openai-introduces-medchembench-for-medicinal-chemistry-reasoning)[\[7\]](https://cdn.openai.com/pdf/09c4ec60-a32e-4652-837b-1811368a3bb0/medchembench-paper.pdf). The structure and reasoning components together carry roughly 73% of the weight, which is why overall scores sit in the low 80s despite the model earning about half credit on the judgment-heavy tasks.

## Structure reconstruction is a gate, not a guarantee

One illustrative task requires assembling a macrocyclic RAS(ON) inhibitor while preserving an 18-atom ring. At xhigh, GPT-6 Astra reconstructed the correct graph and stereochemistry in all ten responses. GPT-5.6 Sol and GPT-5.6 Luna failed all ten times — Sol produced a 15-atom ring, Luna returned two disconnected fragments [\[1\]](https://openai.com/index/introducing-medchembench)[\[7\]](https://cdn.openai.com/pdf/09c4ec60-a32e-4652-837b-1811368a3bb0/medchembench-paper.pdf). Such errors change the molecular identity before any property reasoning begins.

## Predicting competing properties remains hard

A PRMT5 inhibitor problem asks models to predict cellular potency, permeability, efflux and human microsomal clearance for three related compounds. Withheld experiments showed a classic medicinal chemistry tradeoff: compound 23 blocked PRMT5 at lower concentration, while compound 24 crossed the cell barrier more readily and broke down more slowly in the liver-enzyme assay [\[1\]](https://openai.com/index/introducing-medchembench)[\[7\]](https://cdn.openai.com/pdf/09c4ec60-a32e-4652-837b-1811368a3bb0/medchembench-paper.pdf). Models must connect structural changes to multiple, sometimes opposing, readouts — not simply identify the strongest binder.

## Limits acknowledged up front

OpenAI states explicitly that the scores do not establish whether models can discover new drugs or improve laboratory decisions. Every answer was a single attempt without tools, feedback or new experiments. The tasks use historical evidence rather than prospective discovery, and prior exposure to the published sources cannot be ruled out — a heuristic check adding paper identifiers to prompts moved scores by less than one percentage point, with confidence intervals including zero [\[2\]](https://www.unite.ai/openai-introduces-medchembench-for-medicinal-chemistry-reasoning)[\[7\]](https://cdn.openai.com/pdf/09c4ec60-a32e-4652-837b-1811368a3bb0/medchembench-paper.pdf). The paper describes the current benchmark as close to saturation and notes that open-ended design and synthesis grading remains partly tied to reference answers, which may penalize plausible alternatives [\[2\]](https://www.unite.ai/openai-introduces-medchembench-for-medicinal-chemistry-reasoning).

## Proposed extensions

The paper outlines several directions: prospective cases, expert adjudication of alternative answers, explicit treatment of assay uncertainty, and an agent-based track allowing retrieval, calculation, structure verification and multistep revision [\[2\]](https://www.unite.ai/openai-introduces-medchembench-for-medicinal-chemistry-reasoning)[\[7\]](https://cdn.openai.com/pdf/09c4ec60-a32e-4652-837b-1811368a3bb0/medchembench-paper.pdf). That last extension would test how models perform when they must gather evidence and revise over multiple steps — closer to how AI would actually be used in a lab.

For now, MedChemBench provides a concrete, testable measure in a domain where hype has outpaced evidence. It separates chemical comprehension from demonstrated drug-discovery impact, and its component scores identify specific weaknesses in compound selection and quantitative forecasting rather than certifying a model to direct the next experiment.

## Sources

1. [Can AI help scientists identify promising drug candidates?](https://openai.com/index/introducing-medchembench) (openai.com)
2. [OpenAI Introduces MedChemBench for Medicinal-Chemistry Reasoning](https://www.unite.ai/openai-introduces-medchembench-for-medicinal-chemistry-reasoning) (unite.ai)
3. [OpenAI Launches MedChemBench for AI Drug Discovery](https://emergent.sh/news/openai-launches-medchembench-ai-drug) (emergent.sh)
4. [OpenAI MedChemBench: Medicinal Chemistry Tasks, Scores and Limits](https://explainx.ai/blog/openai-medchembench-medicinal-chemistry-benchmark-november-2026) (explainx.ai)
5. [OpenAI releases chemistry benchmark showing AI's uneven ...](https://superpowerdaily.com/posts/openai-releases-chemistry-benchmark-showing-ai-s-uneven-drug-research-skills) (superpowerdaily.com)
6. [OpenAI MedChemBench: GPT-6.2 Astra Scores 84.5%](https://cellcog.ai/blog/openai-medchembench) (cellcog.ai)
7. [https://cdn.openai.com/pdf/09c4ec60-a32e-4652-837b-1811368a3bb0/medchembench-paper.pdf](https://cdn.openai.com/pdf/09c4ec60-a32e-4652-837b-1811368a3bb0/medchembench-paper.pdf) (cdn.openai.com)
8. [Can AI help scientists identify promising drug candidates? | OpenAI](https://openai.com/index/introducing-medchembench?ref=explainx) (openai.com)
