GPT-6 Astra is surprisingly good at ErdosBench math problems

GPT-6 Astra is currently topping ErdosBench for open mathematics problems, despite OpenAI chief scientist Jakub Pachocki stating that math was not a priority for this release. The result suggests OpenAI is shifting focus toward recursive self-improvement and alignment rather than raw reasoning benchmarks, producing "spiky" capabilities that may reflect pattern matching from high-quality training sets rather than genuine reasoning. The article recommends a verification loop that forces the model to output structured step-by-step proofs and self-critique before final answers.

GPT-6 Astra is surprisingly good at ErdosBench math problems GPT-6 Astra is currently topping ErdosBench for open mathematics problems, which is a weird result considering Jakub Pachocki explicitly stated that math wasn't a priority for this release. OpenAI seems to be shifting focus toward recursive self-improvement and alignment rather than raw reasoning benchmarks. This suggests we are hitting a "spiky" development phase where models gain extreme capabilities in specific niches by accident or through side effects of larger training goals, rather than a linear increase in general intelligence. Why the ErdosBench result is confusing Usually, when a model jumps in math performance, it is because of a specific push toward synthetic data generation or a new RLHF pipeline targeting STEM. But if the chief scientist says math wasn't the goal, we have to ask where this performance is actually coming from. In my experience testing these versions, "accidental" leaps usually happen when the model gets better at following complex, multi-step instructions in general, which then happens to translate well to formal proofs. The risk here is that if the model isn't specifically optimized for math, the "correct" answers on ErdosBench might be the result of pattern matching from high-quality training sets rather than a genuine breakthrough in reasoning. If you are using Astra for heavy lifting in LaTeX or complex proofs, I'd suggest double-checking the logic steps because "spiky" performance often means the model can solve a PhD-level problem but then fail a basic arithmetic check in the same session. The shift toward recursive self-improvement If OpenAI is deprioritizing targeted math optimization, they are likely betting on the model's ability to refine its own weights or generate its own training data. This is the "recursive" part. When a model starts improving itself, you stop seeing a smooth curve of progress across all benchmarks. Instead, you get these random spikes. For those of us building on these models, this means the prompt engineering game is changing. We can't assume that a model is "smarter" across the board just because it hit a new high score on a specific benchmark. You might find that GPT-6 Astra handles a specific type of recursive logic perfectly but struggles with a task that GPT-4o handled fine. How to verify the "spiky" performance If you are testing Astra on complex logic, don't trust the final answer. I recommend using a verification loop. Since the model is showing strength in ErdosBench, try forcing it to output its reasoning in a structured format like JSON or a specific logic chain before the final answer. { "step by step proof": "detailed logic here", "verification check": "self-critique of the proof", "final result": "answer" } If the model is just pattern matching, the "verification check" will often be generic or fail to find the error in the "step by step proof". If it is actually reasoning, it should be able to catch its own hallucinations. The cost of these tokens adds up, but it's the only way to tell if you're dealing with a reasoning leap or just a very good guess. Next Microsofts new school AI privacy rules are a reaction to bans → /en/threads/9120/ All Replies （3） I'm skeptical since it keeps hallucinating on basic calculus in my tests. Does this actually hold up on the 402-series tests? Finally I stopped fighting with the old model on my group theory homework. Does this actually hold up for the 700-series? I want to try this tonight. I wonder if this is just a fluke or if it actually handles the 12-step proofs in Wolfram?