AI is learning clinical judgment by practicing on simulated patients

A new reinforcement learning method called ResidencyRL, trained on 57,453 simulated patient encounters, reduced missed red flags from 45.5% to 31.5% and earned an 87.6% expert preference over a baseline model in clinical consultations. The study, submitted on August 7 by researchers using Gemini 3.5 Flash as the starting model, showed the strongest gains in management reasoning, with scores rising from 80.07% to 88.41% on 120 AMIE Mx scenarios, but experts cautioned the results do not validate an autonomous doctor.

All field notes / field-notes FIELD NOTES / CLINICAL AI AI is learning clinical judgment by practicing on simulated patients. ResidencyRL trained a model across complete virtual consultations instead of isolated medical questions. It asked better questions, missed fewer red flags, and earned a strong expert preference. The same study also shows why this is not yet an autonomous doctor. 57,453 training scenarios 45.5 → 31.5% missed red flags 87.6% expert preference versus the baseline model 01 / THE PROBLEM Knowing medicine is not the same as conducting a consultation. A medical exam gives the model a question and the facts needed to answer it. A real consultation does not. The clinician must decide what to ask next, notice what is missing, revise a diagnosis, choose a safe action, explain it, and know when to escalate. That makes clinical reasoning a sequence problem . One early assumption can change every later question. One missed red flag can make an otherwise fluent answer unsafe. ResidencyRL https://arxiv.org/abs/2608.07418 , submitted on August 7, tests whether reinforcement learning across complete simulated encounters can improve this process. The starting point was Gemini 3.5 Flash. The training environment made it practice, receive structured feedback, and try again. facts → answer The relevant information is already present. ≠ ask → update → act → verify The next useful fact must be discovered. 02 / THE SIMULATED RESIDENCY Practice happens inside a generated clinical world. The model is rewarded for the quality of the whole encounter, not only its final diagnosis. - 01 Generate a case A scenario defines the condition, history, required questions, red flags, management, and safety constraints. - 02 Simulate the patient An LLM patient reveals information through dialogue and can use complex or adversarial behavior. - 03 Run the encounter The policy model asks questions, updates its view, uses tools, proposes management, and writes documentation. - 04 Score the trajectory Structured judges evaluate diagnosis, management, intake, communication, documentation, style, and safety. - 05 Reinforce better behavior The training update makes stronger complete trajectories more likely in the next round. 03 / WHAT IMPROVED The strongest gain was a better search for missing information. Held-out adversarial encounters tested whether the model would close too early on a diagnosis or overlook a safety signal. +8.34 Management reasoning On 120 AMIE Mx scenarios, the score rose from 80.07% to 88.41%. Communication rose by 8.50 points and clinical skills by 5.26. 42.9% Oncology comparison wins Across 300 unseen oncology cases, ResidencyRL won 42.9% of composite comparisons versus 18.6% for the baseline. The rest were ties. ↗ Promising, not conclusive AgentClinic and CRAFT-MD results generally moved in the favorable direction, but most individual gains were not statistically significant. 04 / WHAT THE EXPERT PREFERENCE MEANS Clinicians saw a better consultation. They did not validate an AI replacement. preferred for completeness of information gathering preferred for management-plan appropriateness preferred for diagnostic assessment preferred or tied for management-plan safety 05 / THE THREE-MONTH FRONTIER Medical AI is assembling a larger capability stack. Only research published from May 12 through August 12, 2026 is used here. Practice complete consultations ResidencyRL Remember across three visits AMIE disease management Use images, ECGs, and documents Multimodal AMIE See and speak in real time AMIE Video Act inside a simulated EHR MIRA Simulate possible patient responses SepsisAgent Video closes one gap and reveals others. An August 10 AMIE Video study https://arxiv.org/abs/2608.09861 combined low-latency dialogue, clinical planning, and audio-visual perception. Actor-based evaluation was strong, while fine anatomical precision, subtle affect, and fast movement remained difficult. Longitudinal care needs memory and evidence. A June Nature study https://www.nature.com/articles/s41586-026-10764-5 tested management across three visits with persistent state and guideline retrieval. Its authors explicitly say the system is not ready for clinical care. World models try to add consequences. SepsisAgent https://arxiv.org/abs/2605.14723 proposes a treatment, simulates a predicted response, and refines the plan. It is an important direction, but retrospective off-policy value is not a real patient outcome. 06 / WHERE THE EVIDENCE STOPS The evaluator, simulator, and real world are three different things. Simulated patients The environment is text-only, English, and oriented toward US telehealth. It does not include physical examination, delayed lab results, procedures, real provider coordination, or ordinary clinical disruption. Evaluator bias The paper finds that automated judges systematically favored the trained model more strongly than expert clinicians did. Training against a judge can improve the judge's score without closing every clinical gap. No patient outcomes The study measures behavior in simulations and actor-based review. It does not show lower morbidity, safer treatment, better adherence, or improved health in real patients. Limited reproduction Proprietary infrastructure and incomplete optimization details prevent an independent team from reproducing the full training result from the paper alone. A model can only learn the situations its simulator can create and the qualities its evaluator can recognize. Supervised, bounded assistance - structured history collection - clinician preparation - guideline-grounded review - documentation - second-pass red-flag checks Autonomous clinical care - independent diagnosis - unsupervised treatment - safe real-world escalation - better patient outcomes - general performance across settings THE BOTTOM LINE This is evidence that practice can create capability. It is not evidence that simulation has solved medicine. ResidencyRL makes a precise advance. It turns clinical reasoning from a collection of answers into a behavior that can be practiced across long sequences. The model became more complete, more cautious about red flags, and more useful than its baseline in blinded expert review. The result also exposes the next bottleneck. Better simulation can create more practice, but only the real world can show whether the learned behavior improves care. Progress now depends on realistic environments, independent evaluation, prospective studies, and human accountability. 07 / RECENT SOURCES Every current claim uses the latest three-month window. Evidence cutoff: August 12, 2026. Window: May 12 through August 12, 2026. Preprints are labeled and interpreted more cautiously than peer-reviewed studies. ResidencyRL: Reinforcement Learning in Simulated Clinical Environments https://arxiv.org/abs/2608.07418 Aug 7, 2026 · primary preprint Towards Expert-level Medical AI for Real-time Video Consultations https://arxiv.org/abs/2608.09861 Aug 10, 2026 · primary preprint Toward a test of medical AI superintelligence https://www.nature.com/articles/s41591-026-04539-8 Jul 27, 2026 · Nature Medicine perspective AI agents in clinical practice: an evidence map https://www.nature.com/articles/s41746-026-02960-4 Jul 13, 2026 · npj Digital Medicine perspective Towards conversational artificial intelligence for disease management https://www.nature.com/articles/s41586-026-10764-5 Jun 17, 2026; version of record Jul 22 · Nature Agents AMIE and MIRA advance medical AI capabilities https://www.nature.com/articles/d41591-026-00034-2 Jun 30, 2026 · Nature Medicine research highlight Towards autonomous medical artificial intelligence agents https://www.nature.com/articles/s41586-026-10675-5 Jun 17, 2026 · Nature Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policies https://arxiv.org/abs/2606.16721 Jun 15, 2026 · review preprint ClinicalMC: A Benchmark for Multi-Course Clinical Decision-Making https://arxiv.org/abs/2606.03157 Jun 2, 2026 · benchmark preprint Advancing conversational diagnostic AI with multimodal reasoning https://www.nature.com/articles/s41591-026-04371-0 May 14, 2026 · Nature Medicine Agentifying Patient Dynamics through a Clinical World Model https://arxiv.org/abs/2605.14723 May 14, 2026 · primary preprint