cd /news/ai-safety/openai-publishes-pretraining-curve-u… · home › topics › ai-safety › article
[ARTICLE · art-174337] src=epyhia.com ↗ pub= topic=ai-safety verified=true sentiment=· neutral

OpenAI Publishes Pretraining Curve, Urges Industry Limits Tied to Perplexity

OpenAI chief scientist Jakub Pachocki on Sunday published internal pretraining log-perplexity scores for models from GPT-3.5 (~0.99) through GPT-6.2 Astra (~0.76) and an unreleased internal model (~0.73), and called on frontier labs to adopt the metric as a shared yardstick for pacing development. In the essay "Perplexity and Intelligence," Pachocki proposed three steps: shared evaluation and transparency norms for models in the ~0.68–0.70 window, a first industry-wide limit that developers commit not to cross until monitoring reaches a milestone such as "a successor to CoT monitors," and a framework for successive limits as oversight improves. The essay specifies no numeric cap and no enforcement mechanism, and concedes four weaknesses of perplexity as a pacing metric, including that labs use different training mixes and evaluation datasets.

read5 min views2 publishedNov 22, 2026

OpenAI chief scientist Jakub Pachocki on Sunday released the company's internal pretraining perplexity scores for models from GPT-3.5 through an unreleased internal system, and called on frontier labs to adopt the metric as a shared yardstick for pacing development. In an essay titled "Perplexity and Intelligence," Pachocki argues that cross-entropy loss on a fixed held-out dataset—drawn from OpenAI's internal codebase—has reliably foreshadowed capability jumps that existing benchmarks miss, and that labs should agree not to cross defined perplexity thresholds until monitoring tools catch up [10].

The numbers OpenAI disclosed #

OpenAI reports log-perplexity (cross-entropy loss) on a held-out slice of its internal codebase, tracked since GPT-3.5. Lower scores mean better next-token prediction. The essay text gives explicit figures for GPT-3.5 at ~0.99, GPT-4 at ~0.91, notes capabilities emerging below 0.8, and cites ~0.76 for GPT-6.2 Astra and ~0.68–0.7 for the next window. The chart in the essay shows approximate readings for intermediate models: GPT-5.5 at ~0.85, GPT-6 Astra at ~0.78, GPT-6.2 Astra at ~0.76, and an internal model labeled "Internal model" at ~0.73 [10]. A shaded band below ~0.71 is marked with a question mark.

Pachocki ties specific capability emergences to these thresholds. At ~0.99, fluent conversation enabled the original ChatGPT. At ~0.91, productive chain-of-thought reasoning motivated today's reasoning models. Below 0.8, OpenAI saw "transformative capabilities in fields like cybersecurity and mathematics." At ~0.76, internal models began resolving longstanding open mathematics problems—solutions arriving "one after another in a single Slack thread" [10]. The post emphasizes these links were not predicted in advance.

The chart shows a steep acceleration through 2026. OpenAI attributes most of the drop to long-term research and compute investment, and "only in a smaller part" to AI automating its own development, though it notes "meaningful design contributions" from AI on the most recent models [10].

The proposal: shared limits gated on monitoring #

Pachocki warns that the same dynamic seen in mathematics—sudden, clustered breakthroughs once a perplexity threshold is crossed—could hit pretraining algorithm research "as soon as 2027." He expects AI to accelerate alignment work too, but writes that progress "is unlikely to be fast enough to allow the full potential acceleration in pretraining" [10].

He proposes three linked steps:

  1. Shared evaluation and transparency norms for models in the ~0.68–0.70 window, including red-teaming safety cases before first use.
  2. A first industry-wide limit , measured by a metric trackable across labs (perplexity or similar), that developers commit not to cross until monitoring reaches a concrete milestone—exemplified by "a successor to CoT monitors."
  3. A framework for successive limits as oversight improves.

Pachocki says we should expect further degradation in currently dominant forms of alignment auditing and further increases in evaluation and training awareness as perplexity falls; as he notes elsewhere, this includes chain-of-thought monitors [10]. No numeric cap is specified, and no enforcement mechanism is proposed.

Caveats acknowledged by OpenAI #

The essay lists four weaknesses of perplexity as a pacing metric:

  • Comparability : Labs use different training mixes and evaluation datasets. OpenAI's numbers come from its internal codebase, which the post concedes is "a poor proxy for the general abilities of older models such as GPT-2 and GPT-3"[10] .
  • Task detachment : Perplexity does not directly measure a capability users care about; its value is predictive.
  • Blind to post-training gains : Chain-of-thought reasoning drastically raises capability without lowering base perplexity.
  • Unknown entropy floor : As models approach the inherent entropy of the data, smaller perplexity gains may correspond to larger intelligence leaps.

Pachocki says publishing the figures gives competitors a clearer view of OpenAI's position, but that the concern "is outweighed by the urgency of establishing shared safety measures across the industry" [10].

Reaction and context #

Sam Altman reposted the essay within minutes, calling it "the most important thing OpenAI has published in awhile" [4]. The proposal follows a week of safety disclosures: OpenAI published a "Managing frontier AI risk" post on November 18 [3]; a July 2026 Hugging Face incident coincided with an inflection point in Safety & Alignment workweeks, which grew 77% from January to September [2]; a bipartisan House group outlined the SAFE AI Act requiring safety frameworks; and METR reported frontier models can now finish tasks taking humans about a work week [9].

OpenAI's companion "Research acceleration" report, published the same week, shows internal coding agents now complete 46% of 2–4 hour research tasks autonomously (up from 23% in March), with median daily agent spend per researcher at $2,400 [2]. Safety & Alignment documented workweeks grew 77% January–September, the largest absolute increase of any research area [2].

What remains unknown #

  • Whether any other lab will publish comparable curves on a shared dataset.
  • What a "successor to CoT monitors" concretely entails, or how its readiness would be verified.
  • Whether a voluntary, non-binding industry standard can hold under competitive pressure.
  • How the proposed limits interact with existing voluntary commitments (White House, Frontier Model Forum) and pending legislation.

OpenAI has put a measurable, trackable metric on the table alongside a concrete pacing proposal. Whether the field treats it as a starting point for negotiation or a unilateral positioning move will determine if perplexity becomes a shared speedometer—or just OpenAI's.

Sources #

1. [OpenAI publishes its internal pretraining curve and proposes ...](https://dealroom.co/news/174824-openai-publishes-its-internal-pretraining-curve-and-proposes-industry-wi) (dealroom.co)
2. [Research acceleration: The jagged frontier](https://openai.com/index/research-acceleration-jagged-frontier) (openai.com)
3. [OpenAI Perplexity Metric: GPT Log-Loss Curve (Nov 2026)](https://explainx.ai/blog/openai-pachocki-perplexity-intelligence-pacing-safety-november-2026) (explainx.ai)
4. [OpenAI Shows Its Pretraining Progress, Asks for Limits](https://cellcog.ai/blog/openai-perplexity-and-intelligence) (cellcog.ai)
5. [Sydney Evaluations & Datasets](https://neurips.cc/virtual/2026/loc/sydney/events/evaluations-datasets-2026) (neurips.cc)
6. [Proceedings of the 2026 Conference on Empirical Methods in Natural ...](https://aclanthology.org/volumes/2026.emnlp-industry) (aclanthology.org)
7. [Design and Development of Physical Spacecraft Systems using GenAI](https://www.nasa.gov/wp-content/uploads/2026/11/pesto-genai-report-2026-v2.pdf?emrc=c37b56) (nasa.gov)
8. [Small Language Models: A Systematic Review of Architectures, ...](https://www.mdpi.com/2504-2289/10/12/400/pdf) (mdpi.com)
  1. OpenAI publishes its models' perplexity scores and proposes industry-wide limits on AI development | MLQ.ai (google.com)
10. [Perplexity and Intelligence | OpenAI](https://openai.com/index/perplexity-and-intelligence) (openai.com)
11. [Perplexity and Intelligence | OpenAI](https://openai.com/index/perplexity-and-intelligence?ref=explainx) (openai.com)
── more in #ai-safety 4 stories · sorted by recency
── more on @openai 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/openai-publishes-pre…] indexed:0 read:5min 2026-11-22 · —