The core of the problem is that natural language is too ambiguous for complex math. When an AI tries to solve a hard geometry problem in English, it often makes a "leap of faith" that looks correct but is logically bankrupt. The breakthrough comes when we force the AI to work within a formal language—like Lean or Isabelle—where every single step must be mathematically verified by a kernel. If the kernel says the step is wrong, the AI can't just "convince" the user it's right; it has to backtrack and try a different logical path.
For those trying to build a similar AI workflow for technical tasks, here is a practical tutorial on how to implement a basic verification loop:
-
Define the Formal Target: Instead of asking for a solution in text, require the output in a formal language. For example, use a Lean 4 snippet to define the theorem you are trying to prove.
-
Implement the Generator: Use a high-reasoning model (like Claude 3.5 Sonnet or GPT-4o) to generate a candidate proof.
-
The Verification Gate: Pass that output through a compiler or formal verifier.
lean .my_proof.lean
4.Error Feedback Loop: If the compiler returns an error, feed the exact error message back into the LLM. This creates a self-correcting loop where the AI learns from the formal constraints of the math, not just the patterns in its training data.
Standard LLM approach: Probability-based guessing, high hallucination rate in multi-step logic, looks plausible but often fails.Formal Verification approach: Deterministic correctness, zero tolerance for logical gaps, slower generation but 100% reliability once verified.
This shift proves that the next era of LLM agents isn't about bigger datasets, but about better "guardrails" for reasoning. By combining the creative intuition of a model with the rigid laws of formal mathematics, we are seeing AI solve problems that were previously thought to require human-level insight. It's a reminder that the most effective prompt engineering often comes from people who aren't afraid to break the "standard" way of doing things.
LLMs are just massive pattern libraries for math proofs 3h ago
Claude Code actually solved a math problem I'd been stuck on for 4d ago
Next Two American Airlines 737s nearly collided because they were →
these AI tool field notes, with plenty of directly applicable cases.