cd /news/large-language-models/diagtrace-an-interactive-benchmark-f… · home › topics › large-language-models › article
[ARTICLE · art-171562] src=machinebrief.com ↗ pub= topic=large-language-models verified=true sentiment=· neutral

DiagTrace: An Interactive Benchmark for Clinical Decision Making

Researchers introduced DiagTrace, a source-grounded benchmark built from PubMed Central case reports that tests whether large language models can reconstruct a documented diagnostic trajectory through interactive information seeking rather than receiving all patient data upfront. DiagTrace contains 510 cases across eight specialties, with 225 used to evaluate four LLMs on clinical-course alignment, interaction quality and efficiency; results reveal trade-offs in coverage, source-consistent ordering and turn use.

recommendation

Evaluate diagnostic LLMs on incremental, interactive evidence gathering with DiagTrace-style event banks and clinical courses, and report coverage, ordering and turn use together rather than final-answer accuracy alone.

more in playbook: Evals and testing for LLM apps
by read1 min views4 publishedNov 18, 2026

arXiv:2611.22009v1 Announce Type: new Abstract: Large language models (LLMs) have shown strong performance on medical question answering and diagnostic reasoning benchmarks. However, existing evaluations typically provide all patient information upfront and assess only the final diagnosis or rationale. This setup ignores the need to acquire diagnostic evidence incrementally from incomplete information in real-world diagnosis. To bridge this gap, we introduce DiagTrace, a source-grounded benchmark for evaluating whether LLMs can reconstruct the documented diagnostic trajectory through interactive information seeking. Each benchmark instance is built from a PubMed Central case report and includes an initial presentation, an event bank, a clinical course, and diagnostic references. DiagTrace contains 510 cases across eight specialties, with 225 used to evaluate four LLMs on clinical-course alignment, interaction quality, and efficiency. Results reveal trade-offs in coverage, source-consistent ordering, and turn use.

── more in #large-language-models 4 stories · sorted by recency
── more on @diagtrace 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/diagtrace-an-interac…] indexed:0 read:1min 2026-11-18 · —