arXiv:2611.23456v1 Announce Type: new Abstract: Automatic generation of mathematics assessment items is a growing area in educational artificial intelligence, yet classroom deployment remains challenging. Generated questions must satisfy multiple constraints: mathematical correctness, curriculum alignment, appropriate cognitive difficulty, and educator reviewability. While prior work has advanced text generation, reliable assessment authoring requires granular control over localized examination formats and expert educational judgment. To address this challenge, we present VeriMath, a generalized, human-in-the-loop framework for generating rigorous, curriculum-constrained mathematics assessment drafts. Using annotated reference items, the framework anchors generation to the requested examination section and cognitive reasoning level, then applies sequential automated verification and guided repair before submitting drafts for expert review. We evaluate VeriMath through a national curriculum case study using 4,555 mathematics items from the Vietnamese Grade 12 curriculum and 50 distinct generation requirements covering multiple-choice, true/false, and short-answer formats. Across two model backbones, the framework produces verifier-accepted drafts for 94% (47/50) of requirements per run. Blind expert evaluations confirm high mathematical accuracy and curriculum alignment, with strict teacher-readiness assessments accepting up to 56.8% of generated items. Analysis of rejected drafts reveals persistent domain-agnostic challenges involving formatting, statement validity, and solution-step completeness. These findings demonstrate that VeriMath can bridge generative AI and dependable educational applications by producing highly reviewable assessment drafts within tightly constrained national curriculum frameworks.
VeriMath: Human-in-the-Loop Generation and Verification of Curriculum-Aligned Mathematics Assessment Items
A human-in-the-loop framework called VeriMath produced verifier-accepted mathematics assessment drafts for 94% (47/50) of generation requirements per run across two model backbones, according to an arXiv paper evaluating the system on 4,555 items from the Vietnamese Grade 12 curriculum. Blind expert evaluations confirmed high mathematical accuracy and curriculum alignment, but strict teacher-readiness assessments accepted only up to 56.8% of generated items, with rejected drafts failing on formatting, statement validity, and solution-step completeness. The 50 generation requirements spanned multiple-choice, true/false, and short-answer formats.
Pair automated verifier acceptance with blind expert teacher-readiness review before deploying generated assessment items; in this study verifier pass rates of 94% still left only 56.8% of items accepted by strict human review.
more in playbook: Evals and testing for LLM appsRun your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.