The gap between retrieval and reasoning #
If you give a model a problem that has been discussed on MathStackExchange or appears in a textbook, the result is instantaneous and flawless. But the second you introduce a slight, logically sound twist that breaks the known pattern, the "reasoning" often collapses. This is the "stochastic parrot" problem applied to formal logic. A human mathematician can encounter a completely novel problem and use a small set of axioms to build a path to a solution. An LLM needs a path to have been trodden by someone else in its training data to navigate it reliably.
To get around this, we have to shift toward a more robust AI workflow. Pure prompt engineering isn't enough; we need the models to interact with formal verification systems.
Moving toward a real-world AI workflow #
If you want to actually use LLMs for math without getting hallucinated results, you have to move away from the chat box and toward a deployment that involves a feedback loop. Here is a practical approach to structuring this:
-
Formalization: Use the LLM to translate a natural language problem into a formal language like Lean or Coq.
-
Iterative Proving: Let the model propose a proof step.
-
Verification: Pass that step through the formal verifier. If the verifier throws an error, feed that specific error message back into the LLM.
-
Correction: The LLM uses the error log to adjust its "memory" of the path and tries a different tactical approach.
-- Example of a simple property in Lean that a model might attempt
theorem add_comm_example (n m : nat) : n + m = m + n :=
begin
induction n with n hn,
{ rewrite [add_zero, zero_add], exact hn },
{ rewrite [add_succ, succ_add], apply hn },
end
This loop transforms the LLM from a "guessing machine" into a proposal engine for a system that actually understands logic. When we combine LLM agents with symbolic AI, we stop relying on the model's memory and start leveraging its ability to explore a search space quickly. The goal shouldn't be to make the AI "smarter" at math, but to build a system where the AI's vast memory is constrained by rigid, mathematical truth.
Claude Code actually solved a math problem I'd been stuck on for 3d ago
Unreleased OpenAI model takes on 10 major math problems — first 14d ago
Next Qwen 3.8 27B actually beats the larger 3.7 Plus in coding →