Less than 24 hours after OpenAI posted over 700 preprints describing major math results by an internal AI model, the company has withdrawn three of the manuscripts for errors.
On Oct. 6, OpenAI published 722 preprints in a GitHub repository about purported progress in solving 372 math problems in geometry, computer science, algebra and other subjects. The company called the release an effort “to push the frontier of human knowledge and enable further progress in mathematics.”
But on Oct. 7, OpenAI announced the withdrawal of three manuscripts because of a sign error. The error invalidated an argument in one manuscript and the construction used by two dependent papers, according to the announcement. Each of the withdrawn papers carry notices explaining “the gap,” and that OpenAI has “revised 14 other manuscripts with proof repairs, corrected statements, clearer hypotheses and dependencies, and one correction to an obsolete citation.”
That an error cascaded into three of the OpenAI manuscripts is not surprising, said Alex Townsend, an associate professor of mathematics at Cornell University in Ithaca. He suspects there will be more errors found, he said.
“Given the skepticism of AI in the mathematics community, I think that OpenAI should have announced the manuscripts that were lean verified first,” Townsend told us, referring to Lean, the open-source programming language that checks math proofs. “They could have made a separate announcement for the other ones and ask for help verifying them. This would have been more clear to the community and general public.”
A spokesperson for OpenAI told us the company identified the errors during an audit. As with “other research manuscripts,” the process is iterative, the spokesperson said.
“We welcome scrutiny and feedback from the mathematical community,” the spokesperson told us “Where errors are identified, we will work to correct them promptly and withdraw papers if no fixes can be found. We want to give the community time and space to assess the work, and we’re continuing to improve the clarity and presentation of future papers.”
In a post on X, Dan Roberts, a research lead for the company, announced the withdrawals and wrote OpenAI will “continue to update the repo with new formalizations and with any errata we notice.”
The OpenAI spokesperson told us the company’s collaborators at the Advisory Group on Mathematics and Artificial Intelligence recommended releasing the results without waiting for “full formalization,” and that roughly 50% of the results were released unconfirmed.
“We hope to learn from feedback on this release and work with the AGMAI to understand if they have updated recommendations for release protocol moving forward,” the spokesperson said in an email. “Our intent is to follow their protocol as best we can.”
Andrew Sutherland, a senior research scientist in the department of mathematics at the Massachusetts Institute of Technology, said he appreciated OpenAI acting quickly after finding the errors, calling it “the responsible thing to do.”
However, Sutherland emphasized that many mathematicians are “unhappy with how OpenAI has behaved up to this point,” especially in its handling of the Sept. 8 announcement an AI system had solved the Navier-Stokes problem, which asks whether a series of equations describing how gases and liquids move become unreliable at some point.
Following the release, a group of leading math researchers expressed concern that OpenAI rushed the Navier-Stokes announcement “leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others.” More than 8,000 researchers have endorsed the sentiments.
“I think the withdrawals and corrections will be viewed positively by the mathematics community, but it will take a lot more than that to earn back the trust they have lost,” Sutherland told us.
The Association for Human Mathematics criticized the most recent release as well. “Mathematicians did not ask for this work to be done,” the group’s statement, released Oct. 7, said. “Releasing over 700 files at once is not a demonstration of scholarship, but a demonstration of power. We urge mathematicians to discontinue their work with OpenAI and to return to a vision of science that centers human understanding.”
Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on X or Bluesky, like us on Facebook, follow us on LinkedIn, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].