{"slug": "txnqa-benchmarking-analytical-reasoning-and-evidence-grounding-over-transaction", "title": "TxnQA: Benchmarking Analytical Reasoning and Evidence Grounding over Transaction Event Sequences", "summary": "A new arXiv paper introduces TxnQA, an analytical question-answering benchmark of 27 task families (81 templates) plus a 5-scenario fraud-risk reasoning suite built on synthetic, privacy-mitigated merchant transaction data. Evaluating prompted proprietary baselines and six LoRA-adapted open-weight models, the authors report that Gemma-4-31B-IT and Qwen3-8B reach end-to-end correctness scores of 0.867 and 0.816 while evaluated MoE checkpoints reach only 0.632-0.713, and that on the 500-instance fraud suite verdict accuracy frequently exceeds 80% but joint accuracy requiring exact evidence attribution collapses to at most ~23%. The authors conclude that schema compliance, semantic calculation, and evidence attribution are three distinct bottlenecks in transaction-sequence QA.", "body_md": "arXiv:2610.27256v1 Announce Type: new \nAbstract: Analyzing merchant transaction histories requires dynamic temporal filtering, entity resolution, and aggregation over irregular event streams--operations poorly captured by static tabular or time-series forecasting benchmarks. We introduce TxnQA, an analytical QA benchmark comprising 27 task families (81 templates) alongside a 5-scenario fraud-risk reasoning suite that evaluates both verdicts and supporting evidence over synthetic, privacy-mitigated transaction data. Ground-truth answers are computed programmatically, and model performance is decomposed into format-following rate (FFR), conditional content accuracy (CA), and end-to-end correctness (E2E). Evaluating prompted proprietary baselines and six LoRA-adapted open-weight models shows that adaptation reliably enforces output schemas but leaves arithmetic bottlenecks: Gemma-4-31B-IT and Qwen3-8B attain E2E scores of 0.867 and 0.816, whereas evaluated MoE checkpoints reach only 0.632-0.713. Furthermore, on the 500-instance fraud suite, verdict accuracy frequently exceeds 80%, yet joint accuracy (requiring exact evidence attribution) collapses to at most ~23%. These findings identify schema compliance, semantic calculation, and evidence attribution as three distinct bottlenecks in transaction-sequence QA.", "url": "https://wpnews.pro/news/txnqa-benchmarking-analytical-reasoning-and-evidence-grounding-over-transaction", "canonical_source": "https://www.machinebrief.com/news/txnqa-benchmarking-analytical-reasoning-and-evidence-groundi-4x71", "published_at": "2026-10-21 04:00:00+00:00", "updated_at": "2026-10-21 04:50:20.803042+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "machine-learning"], "entities": ["TxnQA", "Gemma-4-31B-IT", "Qwen3-8B", "arXiv", "LoRA"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/txnqa-benchmarking-analytical-reasoning-and-evidence-grounding-over-transaction", "markdown": "https://wpnews.pro/news/txnqa-benchmarking-analytical-reasoning-and-evidence-grounding-over-transaction.md", "text": "https://wpnews.pro/news/txnqa-benchmarking-analytical-reasoning-and-evidence-grounding-over-transaction.txt", "jsonld": "https://wpnews.pro/news/txnqa-benchmarking-analytical-reasoning-and-evidence-grounding-over-transaction.jsonld"}}