GPT-6 Astra Solves 49/50 Unpublished Physics Research Problems Under TheoPhysEval Benchmark For $3 A Problem

A benchmark built by nearly 30 physicists from Harvard, Google DeepMind, TIFR, Duke and the International Centre for Theoretical Sciences found that GPT-6-Astra, run in OpenAI's Codex harness with internet and research tools, solved 49 of 50 unpublished research-level theoretical physics problems on the new TheoPhysEval benchmark, a 98% accuracy rate at an average API cost under $3 per problem. Anthropic's Claude Opus 5.5 in the Claude Code CLI followed at 46/50 (92%), while Gemini 3.8 Flash with Antigravity scored 34/50 (68%), DeepSeek-V4.1-Flash with Codex 27/50 (54%) and GLM 5.3 with Claude Code CLI 21/50 (42%). With web search disabled, GPT-6-Astra and Claude Opus 5.5 both scored 46/50 (92%), which the researchers noted as a small accuracy drop.

After math, it appears that AI is coming after theoretical physics. A new benchmark built by a team of nearly 30 physicists from institutions including Harvard, Google DeepMind, TIFR, Duke and the International Centre for Theoretical Sciences has found that frontier AI agents can now solve genuine, unpublished, research-level theoretical physics problems with near-perfect accuracy, and at a strikingly low cost. The benchmark, called TheoPhysEval, consists of 50 brand-new problems spanning everything from holography and black hole physics to scattering amplitudes, quantum phases of matter and fluid dynamics. Every problem was contributed by a world expert in that specific sub-field, carries a verifiable numerical or symbolic answer, and had never appeared anywhere publicly at the time of testing. These aren’t textbook exercises. The researchers describe them as “turn-the-crank” research problems, the kind where a working physicist takes a known technique and extends it to a new case, work that is often novel enough to be publishable, if not quite groundbreaking. The headline result is the one being talked about: GPT-6-Astra, run inside OpenAI’s Codex harness, solved 49 of the 50 problems when given access to the internet and research tools, a 98% accuracy rate, at an average API cost of under $3 per problem. Anthropic’s Claude Opus 5.5, harnessed in the Claude Code CLI, followed close behind at 46/50, a 92% success rate. The problems were deliberately brutal The researchers asked contributors to hand over the hardest problems they could, with one condition: the answer had to be verifiable but impossible to simply guess. The paper includes a few sample problems to illustrate just how specialized these are. One asks the solver to compute the Page time for a black-string background with two branes in AdS4, a holography problem that requires locating an adjacent five-dimensional calculation in the literature, adapting the equations to four dimensions, and then numerically solving a singular boundary-value problem while being careful around singularities at the endpoints. Another involves constructing a two-parameter Hamiltonian for a fermion parity pump using a “suspension construction” and identifying the central charge of the critical theory at a diabolical point. A third asks for the spanwise length scale of velocity perturbations in flat-plate boundary-layer flow at a specific Reynolds number, solved by setting up and numerically solving the unsteady incompressible Navier–Stokes equations. One of the paper’s own authors, Suvrat Raju of ICTS, noted that it took him personally about a day to work through one of these problems by adapting his own prior published calculation. The paper suggests that before current-generation agents, only a small pool of specialists worldwide would have had both the domain knowledge and the numerical chops to solve several of these at all, and it would have taken them days rather than hours. Astra vs Opus vs the rest of the field The team tested five agent setups across two conditions: a “realistic” track where the agent had full internet and tool access, and an “isolated” track where web search was disabled to test how much of the literature the models had effectively memorized. The results: Realistic track web access allowed : - GPT-6-Astra with Codex: 49/50 98% - Claude Opus 5.5 with Claude Code CLI: 46/50 92% - Gemini 3.8 Flash with Antigravity: 34/50 68% - DeepSeek-V4.1-Flash with Codex: 27/50 54% - GLM 5.3 with Claude Code CLI: 21/50 42% Isolated track no internet access : - GPT-6-Astra with Codex: 46/50 92% - Claude Opus 5.5 with Claude Code CLI: 46/50 92% - Gemini 3.8 Flash with Antigravity: 31/50 62% - GLM 5.3 with Claude Code CLI: 26/50 52% - DeepSeek-V4.1-Flash with Codex: 25/50 50% What stood out to the researchers was how little accuracy dropped when web search was switched off. Since many of these problems require pulling specific formulas straight out of recent papers, a small gap between the two tracks suggests the leading models have effectively absorbed large chunks of the theoretical physics literature into their weights, rather than needing to look things up in the moment. Cost and compute told their own story. Astra’s run was by far the cheapest and most token-efficient of the lot, burning through roughly 37 million tokens on the realistic track for an estimated $116 in total API spend across all 50 problems. Opus, by contrast, consumed around 211 million tokens and cost roughly $240 for the same track, reflecting Opus’s more thorough, higher-token reasoning style. GLM’s developers ran their evaluation on a flat-rate coding plan priced at $180 for both tracks combined, though the paper notes the equivalent API cost would have run north of $300. The one problem nobody cracked was an inverse problem: given a curve, find the autonomous differential equation that produced it. The researchers kept it in the suite specifically because it mirrors something a real researcher might face, messy, underspecified data that has to be reverse-engineered into physics. Why this matters, and why it’s not game over for physicists The paper is careful not to oversell the result as agents replacing physicists. The authors frame problem-solving as just one ingredient in actual physics research, behind asking the right high-level questions and having the “taste” to know which problems are worth pursuing in the first place. They note that when asked to generate new problems themselves, today’s agents tend to fixate on details a human researcher would find uninteresting, and that no agent has yet produced an unprompted breakthrough in the field. This caution echoes a point Harvard physicist Matthew Schwartz made earlier this year, when he used Claude to compress a year-long theoretical calculation into two weeks, while stressing that constant human supervision was still essential and that “AI is not doing end-to-end science yet.” The TheoPhysEval authors go a step further in their concluding discussion, warning that problems in their collection, ones that used to be reasonable starter projects for PhD students, may no longer be appropriate to assign unless the goal is explicitly pedagogical rather than discovery-oriented. The result also lands against the backdrop of a very public race in AI-for-math, where OpenAI recently claimed to have resolved the Navier-Stokes Millennium Prize problem https://officechai.com/ai/openai-shares-solution-to-navier-stokes-problem-created-by-a-model-significantly-more-capable-than-astra/ using an unreleased internal model running roughly 10,000 coordinating agents. More recently, OpenAI dropped https://officechai.com/ai/openai-releases-over-300-mathematical-results-produced-by-an-internal-model/ the proofs of more than 300 open math problems, which has led to much hand-wringing https://officechai.com/ai/openais-300-proofs-are-quality-of-work-easy-rejects-mathematician-sophie-huiberts-in-a-post-shared-by-terence-tao/ by the mathematics community. TheoPhysEval’s authors explicitly draw a contrast with that kind of effort: their benchmark isn’t about AI breaking new ground in physics, it’s a controlled measurement of how well current agents can do the concrete calculational work that sits underneath real research, using ordinary, off-the-shelf tools rather than massive bespoke compute. The full set of 50 problems has been released publicly, and the researchers are hoping it becomes a standard yardstick for how well future models can actually assist, rather than just impress, working theoretical physicists.