STATESCOPE: Interactive, Grounded Diagnosis and Repair of Solution-State Drift in Mathematical Reasoning

Researchers L D M S Sai Teja, Sangam Sai Anish and Akshaj Agadi presented StateScope, an interactive system for detecting, localizing and repairing solution-state drift in mathematical reasoning, at AACL-IJCNLP 2026 in Hengqin, China (pages 82–88). StateScope compares a solver's evolving state against a symbolically verified reference trajectory using a SymPy oracle that checks every transition and explains component-level differences without an LLM judge, and its paired evaluation covers 250 problems and twelve open and proprietary models, alongside a human study measuring diagnostic accuracy and time. The system lets users step through a run, edit an operation at any retained step, repair a model's claimed state and branch repeatedly from earlier interventions, targeting reasoning researchers, model developers and educators who need to inspect more than a final answer.

Abstract Final-answer accuracy hides where a mathematical solution first becomes inconsistent and whether a later correction merely masks that error. We present StateScope, an interactive system for detecting, localizing, and repairing solution-state drift: divergence between a solver’s evolving state and a symbolically verified reference trajectory. Under a controlled comparison, two solvers receive the same Computer Algebra System CAS -derived operation schedule and typed arguments. We provide two systems, 1 System C records the model’s textual state claim, 2 System D re-derives state in an external typed ledger. A SymPy oracle checks every transition and explains component-level differences without an LLM judge. Users can step through a run, edit an operation at any retained step, directly repair System C’s claimed state, continue the model, and repeatedly branch from earlier interventions. Our paired evaluation covers 250 problems and twelve open and proprietary models, and a human study measures diagnostic accuracy and time. StateScope targets reasoning researchers, model developers, and educators who need to inspect and manipulate more than a final answer. - Anthology ID: - 2026.aacl-demo.11 - Volume: - Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations https://aclanthology.org/volumes/2026.aacl-demo/ - Month: - November - Year: - 2026 - Address: - Hengqin, China - Editors: - Ningyu Zhang https://aclanthology.org/people/ningyu-zhang/ , Derry Wijaya https://aclanthology.org/people/derry-tanti-wijaya/ - Venues: - AACL https://aclanthology.org/venues/aacl/ | IJCNLP https://aclanthology.org/venues/ijcnlp/ - SIG: - Publisher: - Association for Computational Linguistics - Note: - Pages: - 82–88 - Language: - URL: - https://aclanthology.org/2026.aacl-demo.11/ https://aclanthology.org/2026.aacl-demo.11/ - DOI: - Cite ACL : - L D M S Sai Teja, Sangam Sai Anish, and Akshaj Agadi. 2026. STATESCOPE: Interactive, Grounded Diagnosis and Repair of Solution-State Drift in Mathematical Reasoning https://aclanthology.org/2026.aacl-demo.11/ . In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations , pages 82–88, Hengqin, China. Association for Computational Linguistics. - Cite Informal : - STATESCOPE: Interactive, Grounded Diagnosis and Repair of Solution-State Drift in Mathematical Reasoning https://aclanthology.org/2026.aacl-demo.11/ Sai Teja et al., AACL-IJCNLP 2026 - PDF: - https://aclanthology.org/2026.aacl-demo.11.pdf https://aclanthology.org/2026.aacl-demo.11.pdf