Researchers introduced DeFAb, a benchmark for defeasible abduction in foundation models, converting knowledge bases into 372,648+ logically verifiable instances. Frontier language models achieved at m…
A user tested Claude's ability to generate Lean 4 code to formalize a ring theorem, achieving a proof after 11 iterations but with five unproven 'sorry' sections. The experiment highlights challenges …
John D. Cook used Anthropic's Claude (Sonnet 4.6 Medium) to find a typo in a blog post about quaternion rotations by asking it to write Lean code verifying the post's theorems. After four iterations, …
Researchers introduced MA-ProofBench, the first formal theorem-proving benchmark for mathematical analysis, containing 200 problems across two difficulty levels. Evaluations of leading LLMs showed poo…
Anthropic's Claude AI generated a Lean formal proof for a Fourier coefficient calculation involving Bessel functions, requiring eight iterations to fix errors and produce a working proof with four 'so…