Paul Arnold
Author
Lisa Lock
Scientific Editor
Robert Egan
Senior Editor
One of the many challenging parts of being a doctor is diagnosing exactly what is wrong with a patient. This is because numerous illnesses often have overlapping symptoms, which can make the diagnostic journey a long one.
In addition to asking patients questions, doctors may need to order tests and consult medical references or even other professionals. Obviously, for the individual patient, the sooner a diagnosis is reached, the sooner they can start receiving the most appropriate treatment.
Now, in yet another breakthrough for AI, Microsoft researchers claim they have developed a tool that can outperform individual generalist physicians in diagnosing complex medical cases.
It is called DxO, which stands for Diagnostic Orchestrator, and simulates a panel of clinical experts who collaborate to reach a diagnosis.
The diagnostic challenge #
To see how well DxO performs and how it compares with medical professionals, researchers developed a testing framework called SDBench, which was essentially an interactive quiz.
The team took 304 published diagnostic cases from the New England Journal of Medicine and 325 emergency department admission records. They turned each of these patient files into a step-by-step test.
At the start of each test, a doctor or DxO was given a summary called a clinical vignette. This included basic details like the patient's age and sex and their main symptoms.
For example, one case started with: "A 29-year-old woman was admitted to the hospital because of sore throat and peritonsillar swelling and bleeding. Symptoms did not abate with antimicrobial therapy." An AI system acted as a gatekeeper, providing information from the original case file in response to questions and test requests. Once a doctor or DxO felt they had enough information, they submitted a final diagnosis. An AI judge graded the diagnosis against the one recorded in the original case file.
The results are in #
The difference between AI and real doctors was striking, as the team reports in a paper published in the journal Science.
Regarding the complex published cases, DxO made an accurate diagnosis 83.2% of the time and achieved 80.3% accuracy on emergency department admission cases. Human doctors, on the other hand, averaged 36.1% and 46.8%, respectively.
There were also clear differences in approach. DxO asked more questions about the patient's background upfront before ordering tests, which could help reduce testing costs.
While AI appears to have come out on top, the researchers state that it doesn't mean it is better than doctors in real-world practice. "These comparisons do not establish superiority over specialists, multidisciplinary teams, or real-world clinical practice." Additionally, the Microsoft team did not conduct the testing in everyday health care settings. "As a result, SDBench measures diagnostic reasoning under idealized information conditions rather than bedside performance or patient outcomes."
The study authors say future work should explore how doctors and AI systems could work together. They also suggest the technology could be used in medical education and training.
Written for you by our author [Paul Arnold](https://sciencex.com/help/editorial-team/paul-arnold/), edited by [Lisa Lock](https://sciencex.com/help/editorial-team/lisa-lock/), and fact-checked and reviewed by [Robert Egan](https://sciencex.com/help/editorial-team/robert-egan/)—this article is the result of careful human work. We rely on readers like you to keep independent science journalism alive.
If this reporting matters to you, please consider a [donation](https://sciencex.com/donate/?utm_source=story&utm_medium=story&utm_campaign=story) (especially monthly). You'll get an **ad-free** account as a thank-you.
Publication details
Harsha Nori et al., Sequential diagnosis with language models, Science (2026). DOI: 10.1126/science.aeh7930 Journal information: New England Journal of Medicine ,
[Science](https://medicalxpress.com/journals/science/)
Key medical concepts
Diagnostic Accuracy
Clinical categories
Family medicine © 2026 Science X Network
Citation: AI gets more diagnoses right than generalist doctors in medical challenge (2026, October 23) retrieved 24 October 2026 from https://medicalxpress.com/news/2026-10-ai-generalist-doctors-medical.html