{"slug": "terminal-style-mathematical-modeling-bench-the-first-agentic-and-verifiable-for", "title": "Terminal-style Mathematical Modeling Bench: The First Agentic and Verifiable Benchmark for Mathematical Modeling", "summary": "Researchers introduced TB-MathModel, a terminal-style benchmark that makes 18 real CUMCM and MCM contest problems from 2023–2025 agentic and verifiable, scoring agents' single JSON artifacts against reproductions of Outstanding-award papers. The benchmark uses direction-aware one-sided scoring and two-layer feasibility verification — a programmatic gate over six decidable invariant forms plus 624 Lean 4 obligations, with a hard gate zeroing infeasible answers — and scores each task against a panel of human outstanding papers. Across seven multimodal models (three seeds each), the strongest reached a multi-reference O-Eval of 64.58%, while a $2.54 open model held 59.81% at 1/100 the cost; source code is at github.com/wxj630/terminal-bench-math-modeling.", "body_md": "arXiv:2611.08237v1 Announce Type: new \nAbstract: Mathematical modeling turns an open-ended real-world problem into assumptions, a model, a solution, and a defended recommendation, and its answers matter only if they are both close to a credible reference and physically feasible; existing benchmarks either score solver fragments or grade reports with an LLM judge, leaving those numbers unchecked. We introduce TB-MathModel (the Terminal-style Mathematical Modeling Bench), the first benchmark to make complete contest-style modeling problems agentic, verifiable, and terminal at once, over 18 real CUMCM and MCM problems (2023--2025). Each task ships the verbatim statement, official attachments, and figures in a containerized terminal; the agent returns one JSON artifact, scored on the numeric core of the final question against a reproduction of the Outstanding-award paper. Two design choices distinguish it: direction-aware, one-sided scoring, which never penalises beating the anchor, and two-layer feasibility verification---a programmatic gate over six decidable invariant forms plus 624 Lean~4 obligations---with a hard gate that zeroes infeasible answers. Each task is scored against a panel of human outstanding papers rather than one, so a model must be close to many accepted answers; across seven multimodal entries (three seeds each) the strongest reach a multi-reference O-Eval of 64.58\\%, while the \\$2.54 open model holds 59.81\\% at $1/100$ the cost. TB-MathModel turns the numeric core of modeling into a reproducible, machine-checkable signal for measuring long-horizon modeling competence and for collecting agent trajectories. The source code is available at https://github.com/wxj630/terminal-bench-math-modeling.", "url": "https://wpnews.pro/news/terminal-style-mathematical-modeling-bench-the-first-agentic-and-verifiable-for", "canonical_source": "https://www.machinebrief.com/news/terminal-style-mathematical-modeling-bench-the-first-agentic-gdmg", "published_at": "2026-11-06 05:00:00+00:00", "updated_at": "2026-11-06 06:20:10.382488+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-research", "ai-agents", "large-language-models"], "entities": ["TB-MathModel", "CUMCM", "MCM", "Lean 4", "GitHub"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/terminal-style-mathematical-modeling-bench-the-first-agentic-and-verifiable-for", "markdown": "https://wpnews.pro/news/terminal-style-mathematical-modeling-bench-the-first-agentic-and-verifiable-for.md", "text": "https://wpnews.pro/news/terminal-style-mathematical-modeling-bench-the-first-agentic-and-verifiable-for.txt", "jsonld": "https://wpnews.pro/news/terminal-style-mathematical-modeling-bench-the-first-agentic-and-verifiable-for.jsonld"}}