cd /news/large-language-models/clinical-reasoning-graphs-structured… · home topics large-language-models article
[ARTICLE · art-61007] src=benchmarklist.com ↗ pub= topic=large-language-models verified=true sentiment=· neutral

Clinical Reasoning Graphs: Structured Evaluation of LLM Diagnostic Reasoning Reveals Competence Without Consistency

Researchers at an undisclosed institution introduced clinical reasoning graphs to evaluate LLM diagnostic reasoning across 750 traces from five models on 50 NEJM cases. They found that while LLMs achieved 60-70% diagnostic accuracy, their reasoning showed no schema-scale consistency, with graph similarity nearly identical for correct and incorrect diagnoses. The findings suggest that accuracy alone is insufficient and process-level evaluation is needed.

read1 min views2 publishedJun 29, 2026

Modern large language models (LLMs) reach 60-70% diagnostic accuracy on complex clinical case benchmarks, but accuracy alone cannot distinguish stable clinically-grounded reasoning from pattern matching. We introduce clinical reasoning graphs, structured graph representations extracted from free-text LLM diagnostic traces using a domain-grounded ontology with 5 node types and 7 edge types. We apply this pipeline to 750 traces from five LLMs across 50 New England Journal of Medicine Clinicopathological Conference cases and three prompt conditions, and test whether diagnostic traces show stable structured reasoning patterns, or diagnostic schemas, for clinically similar cases. We operationalize this as higher graph similarity among clinically similar cases than among clinically dissimilar ones. Across 15 model-condition comparisons, within-cluster and between-cluster composite similarity are nearly equal, and no comparison survives multiple-testing correction; a component-level analysis finds any residual content signal far below schema scale. Graph similarity is also nearly identical for pairs of models that are both correct (0.488) and both incorrect (0.484), suggesting that graph structure captures a dimension not reflected in diagnostic accuracy. Structured reflection prompting increases explicit discriminating-feature analysis within traces (+33%) but does not increase cross-case consistency. These results show diagnostic competence without schema-scale reasoning consistency, and indicate that final-answer accuracy should be complemented by process-level evaluation. We release the ontology, extraction pipeline, validation protocol, and the extracted reasoning graphs and similarity artifacts as resources for structured evaluation of LLM clinical reasoning. Category: Uncategorized. Imported rows: 5. Top imported result: Sonnet 4.5, rank 1, 0.50.

── more in #large-language-models 4 stories · sorted by recency
── more on @new england journal of medicine 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/clinical-reasoning-g…] indexed:0 read:1min 2026-06-29 ·