# Opus 5.5 Formalized CPython in Lean and Surfaced 418 Bugs

> Source: <https://explainx.ai/blog/opus-5-5-cpython-lean-formalization-418-bugs-2026?utm_source=rss&utm_medium=feed&utm_campaign=blog>
> Published: 2026-11-10 00:00:00+00:00

**Claude Opus 5.5 helped build a machine-checked Lean model of CPython 3.15.2 that has already produced 418 bug reports**, David Andrews and Chao Zhang wrote in [Formalizing CPython](https://broyojo.com/python-formalization/?ref=explainx) on **November 9, 2026**. The project is not a benchmark press release: it is a **line-by-line copy** of interpreter functions, **differentially tested** against real Python, then **reviewed by a second agent** before it counts toward progress.

For Python builders, the payoff is practical. You get a public list of **reproducer scripts** for corner-case failures, a reusable **agent workflow** for other codebases, and a sobering lesson on what "formally verified" must mean when **models** can lie even when **proof checkers** do not.

## TL;DR

| Metric | Value (Nov 9 snapshot) | 
|---|---|
| Lean lines | 613,318 across 1,539 files | 
| Functions accepted | 4,071 / 26,004 tracked (15.7%) | 
| Checked theorems | 18,898 lemmas | 
| Proved vs refuted top-level props | 3,834 proved, 588 refuted | 
| Bug reports | 418 with reproducers | 
| Serious reachability | ~55 crash or memory-corruption class | 
| Primary model | Claude Opus 5.5 | 
| Review helper | GPT-6.1 Sol in Codex per Python.org thread | 

## How the pipeline works

Every function passes four steps published on the site:

1. **Copy** — Rewrite C or Python source in Lean, preserving error messages and edge cases.
2. **Test** — Run the Lean copy against CPython on**at least 50,000** generated inputs per function.
3. **Prove** — State a property and machine-check it; if false, keep the counterexample.
4. **Check** — A**second agent** that did not author the copy reviews it and asks whether a**broken** implementation could still pass the proof.

That last gate exists because an early breadth-first pass produced proofs that were **logically valid but semantically empty**. The site's "proofs that prove nothing" section gives the canonical bad example: "if this function returns a number, the number is at least zero" — true even for a function that **always crashes**.

The project started **October 29, 2026** and explicitly credits **[julia-in-lean](https://github.com/MilesCranmer/julia-in-lean?ref=explainx)** as inspiration. Julia-in-lean reported **hundreds** of Base bugs through a similar copy-and-test approach.

## What developers are saying

Andrews opened a [Python.org discuss thread](https://discuss.python.org/t/formalizing-cpython-in-lean-findings-from-a-machine-checked-model-and-how-youd-like-them/109978?ref=explainx) on **November 9, 2026** to coordinate with CPython maintainers before flooding the tracker.

He wrote that the team filed **only six crash reports** initially — each personally verified on **3.15.2 and main** — because automated October flooding made maintainers wary. Examples he named include `bytearray.strip()` use-after-free paths, ElementTree slice assignment reference bugs, and `functools.partial` keyword-merge buffer issues.

The thread is the right place to watch for **maintainer triage** and duplicate linking; Andrews says findings that match existing issues reference those issues instead of re-filing.

## Scale and coverage snapshot

Opus 5.5 CPython Lean formalization project overview with checked theorem counts
*Image: [David Andrews and Chao Zhang](https://broyojo.com/python-formalization/?ref=explainx)*

The landing page tracks coverage by subsystem. At publication:

- **Grammar rules** hit**100%** of reviewed grammar targets.
- **Built-in types** sit around**35%** accepted in the progress chart narrative.
- **Interpreter core** remains early (**~10%** in the visualization copy).
- **Standard library** Python modules are**~9%** , with most pure-Python stdlib not yet formalized.

Multithreading is **out of scope for now**, which matters if you hoped for a concurrency proof story.

## Bug taxonomy, not vanity counts

Chalkboard proof steps for Opus 5.5 CPython Lean formalization bug hunting

The project classifies all **418** reports. Top buckets on the site:

| Category | Count | 
|---|---|
| C and Python implementations disagree | 129 | 
| Other | 122 | 
| Wrong result | 56 | 
| Crash | 28 | 
| Memory corruption | 27 | 
| Doesn't match docs | 26 | 
| Wrong error or message | 18 | 
| Missing input check | 12 | 

Authors stress that **~55** issues can **crash or corrupt memory** using normal Python, often when **user callbacks mutate containers** while C code still holds references. CPython does **not** treat hostile Python as a security boundary, so these are **ordinary correctness bugs**, not CVE-class threat models — but they still break production.

Six exemplar reproducers on the site include:

- **`vars(b).setdefault(Key.NAME, ...)` segfault** with StrEnum keys on attribute dicts.
- **ElementTree backwards slice assignment** duplicating children and dropping references.
- **`a2b_base32` padding ignored** changing results by padding position in 3.15.

Full scripts live in the [GitHub repository](https://github.com/Broyojo/python-formalization?ref=explainx).

## When correct proofs were still wrong

The post documents failures where Lean accepted a theorem but **real Python disagreed**:

- **JSON encoder proofs** assumed allocations never fail, "proving" absurd nesting depths including**16 TB** indentation strings until memory failure was modeled.
- **Pickle depth proofs** missed a**one-time codec import** that adds**nine** hidden stack frames on first use in a fresh process.
- **Set memory reuse** turned an accepted proof into a**real bug** once address reuse was modeled — now on the report list.

That section is the article's core lesson for teams tempted to automate proofs without **ground-truth regression**.

## How this differs from Anthropic's SDK verification story

explainx.ai previously covered [Boris Cherny's Opus 5.5 Lean verification of the Claude Agent SDK](https://explainx.ai/blog/boris-cherny-opus-5-5-lean-tla-formal-verification-sdk-2026): **16 PRs**, **24 bugs**, **1,529 theorems**, **zero sorry placeholders** in that formal pass.

The CPython project is larger in **lines** and **scope**, but smaller in **percent complete**. It also targets **CPython behavior**, not a single vendor SDK — so successful bugs land on **python/cpython**, not one company's repo.

Anthropic's wider formal-methods push — [123-library Rocq work](https://explainx.ai/blog/anthropic-claude-formal-verification-123-libraries-rocq-risc-v-proofs-2026), [Fermat-related Lean experiments](https://explainx.ai/blog/anthropic-claude-fermats-last-theorem-lean-proof-2026), and [OpenAI's Lean RL environments](https://explainx.ai/blog/openai-math-lean-rl-environments-hugging-face-harbor-369-tasks-2026) — shares tooling themes, but Andrews and Zhang's effort is **independent research**, not an Anthropic product launch.

## What agent builders can reuse

The repository publishes operational artifacts meant for reuse:

- **AGENTS.md** — Instructions for modeling agents.
- **REVIEW_PROMPT.md** — Independent reviewer checklist ("could a broken version pass?").
- **coverage/methods** — Per-function status tracking.

Their guidance generalizes beyond Python:

- Pin **one exact upstream version** .
- Inventory **all functions** up front so progress cannot hide gaps.
- Copy **source** , not manual prose.
- Model **hostile user callbacks** in container APIs.
- **Test before prove** .
- **Re-run real interpreter checks** after proof acceptance.

If you run coding agents on infrastructure code, pair formal passes with **runtime guardrails** so agent-proposed patches cannot ship without human gates. Tools like [AgentBeam](https://agentbeam.com/?ref=explainx) address a different layer — stopping dangerous agent **actions** at execution time — while this project addresses **semantic fidelity** of the interpreter underneath.

## Limits and honest caveats

- **15.7% function acceptance** means most of CPython is still unmodeled; absence of a report is not evidence of correctness.
- **Opus 5.5 wrote most Lean** ; human direction and review still dominate acceptance counting.
- **Six GitHub issues filed** at thread time implies hundreds of reports remain**repository-local** pending maintainer appetite.
- **Concurrent semantics** are explicitly deferred.
- Proofs certify statements about the **Lean copy** , not a blanket certificate for all of Python.

## Reading the bug reports as a Python maintainer

If you maintain libraries on top of CPython, skim the report repository by **category** before chasing individual issues:

- **C vs Python disagree** — Often means fast C paths and Python fallbacks diverge on edge types. Fix strategy is usually**unify semantics** or**document intentional differences** .
- **Wrong error types** — Breaks`except` hierarchies silently. High value for web frameworks parsing headers or email.
- **Memory corruption with callbacks** — Common in**during-loop mutation** patterns; audit C extensions that borrow references without re-fetching.
- **Doc mismatches** — Low severity individually, but agents trained on docs**will hallucinate APIs** if docs lie.

The project's six initially filed crash issues on discuss.python.org are instructive because Andrews personally bisected them on **main** and **3.15.2**. That is the bar for accepting **agent-filed** bugs upstream: human-readable narrative, minimal reproducer, duplicate search, and explicit severity.

Teams using **Opus-class agents** on application code should import the same bar: agents file issues; humans merge only when reproducers survive review.

## Cost and effort signals

The site does not publish dollar spend, but scale hints at **multi-week agent throughput**:

- **613k Lean lines** in under two weeks of calendar time since October 29.
- **50k inputs per function** differential tests multiply CPU hours quickly.
- **Second-agent review** doubles model calls on every accepted function.

For budgeting, treat formalization as **CI plus LLM bills**, not a one-shot prompt. The payoff is **structured defect lists** you can attach to release gates.

Downstream, if you ship **Python-heavy agent frameworks**, these bugs matter because agents generate **weirdly valid** programs that stress rarely used stdlib corners. Pair formal hunts with fuzzing traditions from [theorem verification enrollments](https://explainx.ai/blog/theorem-oss-verify-free-formal-verification-open-source-fuzz-targets-rocq-how-to-enroll-2026) and runtime sandboxes.

## Try it yourself

Clone [github.com/Broyojo/python-formalization](https://github.com/Broyojo/python-formalization?ref=explainx), read **CONTRIBUTING.md**, and pick a file marked as not yet started from **coverage/methods**. Independent review PRs count the same as new models.

For comparison baselines, see older semantic work cited on the site — λπ, K-framework Python 3.3, JavaScript mechanizations, and CompCert/seL4 as the gold standard for **whole-program** verification.

## Related reading

- [Boris Cherny Opus 5.5 Lean SDK verification](https://explainx.ai/blog/boris-cherny-opus-5-5-lean-tla-formal-verification-sdk-2026) — Vendor SDK formal pass contrast.
- [Anthropic formal verification libraries](https://explainx.ai/blog/anthropic-claude-formal-verification-123-libraries-rocq-risc-v-proofs-2026) — Lab-scale proof programs.
- [OpenAI Lean RL environments](https://explainx.ai/blog/openai-math-lean-rl-environments-hugging-face-harbor-369-tasks-2026) — Training agents for proof tasks.
- [Axiom Stacks Lean algebra project](https://explainx.ai/blog/axiom-stacks-project-lean-16m-lines-ai-agents-algebraic-geometry-apache-2026) — Large math formalization with agents.
- [Theorem OSS verification enrollment](https://explainx.ai/blog/theorem-oss-verify-free-formal-verification-open-source-fuzz-targets-rocq-how-to-enroll-2026) — Community formal verification tooling.
- [Jose Valim on languages in the AI era](https://explainx.ai/blog/jose-valim-programming-languages-ai-era-agent-tooling-2026) — Why runtimes still matter for agents.