All field notes FIELD NOTES / CLINICAL AI
ResidencyRL trained a model across complete virtual consultations instead of isolated medical questions. It asked better questions, missed fewer red flags, and earned a strong expert preference. The same study also shows why this is not yet an autonomous doctor.
57,453 training scenarios
45.5 → 31.5% missed red flags
87.6% expert preference versus the baseline model
01 / THE PROBLEM
Knowing medicine is not the same as conducting a consultation. #
A medical exam gives the model a question and the facts needed to answer it. A real consultation does not. The clinician must decide what to ask next, notice what is missing, revise a diagnosis, choose a safe action, explain it, and know when to escalate.
That makes clinical reasoning a sequence problem. One early assumption can change every later question. One missed red flag can make an otherwise fluent answer unsafe.
ResidencyRL, submitted on August 7, tests whether reinforcement learning across complete simulated encounters can improve this process. The starting point was Gemini 3.5 Flash. The training environment made it practice, receive structured feedback, and try again.
facts → answer
The relevant information is already present.
≠
ask → update → act → verify
The next useful fact must be discovered.
02 / THE SIMULATED RESIDENCY
Practice happens inside a generated clinical world. #
The model is rewarded for the quality of the whole encounter, not only its final diagnosis.
- 01
Generate a case
A scenario defines the condition, history, required questions, red flags, management, and safety constraints.
- 02
Simulate the patient
An LLM patient reveals information through dialogue and can use complex or adversarial behavior.
- 03
Run the encounter
The policy model asks questions, updates its view, uses tools, proposes management, and writes documentation.
- 04
Score the trajectory
Structured judges evaluate diagnosis, management, intake, communication, documentation, style, and safety.
- 05
Reinforce better behavior
The training update makes stronger complete trajectories more likely in the next round.
03 / WHAT IMPROVED
The strongest gain was a better search for missing information. #
Held-out adversarial encounters tested whether the model would close too early on a diagnosis or overlook a safety signal.
+8.34
Management reasoning
On 120 AMIE Mx scenarios, the score rose from 80.07% to 88.41%. Communication rose by 8.50 points and clinical skills by 5.26.
42.9%
Oncology comparison wins
Across 300 unseen oncology cases, ResidencyRL won 42.9% of composite comparisons versus 18.6% for the baseline. The rest were ties.
↗
Promising, not conclusive
AgentClinic and CRAFT-MD results generally moved in the favorable direction, but most individual gains were not statistically significant.
04 / WHAT THE EXPERT PREFERENCE MEANS
Clinicians saw a better consultation. They did not validate an AI replacement. #
preferred for completeness of information gathering
preferred for management-plan appropriateness
preferred for diagnostic assessment
preferred or tied for management-plan safety
05 / THE THREE-MONTH FRONTIER
Medical AI is assembling a larger capability stack. #
Only research published from May 12 through August 12, 2026 is used here.
Practice complete consultations ResidencyRL
Remember across three visits AMIE disease management
Use images, ECGs, and documents Multimodal AMIE
See and speak in real time AMIE Video
Act inside a simulated EHR MIRA
Simulate possible patient responses SepsisAgent
Video closes one gap and reveals others.
An August 10 AMIE Video study combined low-latency dialogue, clinical planning, and audio-visual perception. Actor-based evaluation was strong, while fine anatomical precision, subtle affect, and fast movement remained difficult.
Longitudinal care needs memory and evidence.
A June Nature study tested management across three visits with persistent state and guideline retrieval. Its authors explicitly say the system is not ready for clinical care.
World models try to add consequences.
SepsisAgent proposes a treatment, simulates a predicted response, and refines the plan. It is an important direction, but retrospective off-policy value is not a real patient outcome.
06 / WHERE THE EVIDENCE STOPS
The evaluator, simulator, and real world are three different things. #
Simulated patients
The environment is text-only, English, and oriented toward US telehealth. It does not include physical examination, delayed lab results, procedures, real provider coordination, or ordinary clinical disruption.
Evaluator bias
The paper finds that automated judges systematically favored the trained model more strongly than expert clinicians did. Training against a judge can improve the judge's score without closing every clinical gap.
No patient outcomes
The study measures behavior in simulations and actor-based review. It does not show lower morbidity, safer treatment, better adherence, or improved health in real patients.
Limited reproduction
Proprietary infrastructure and incomplete optimization details prevent an independent team from reproducing the full training result from the paper alone.
A model can only learn the situations its simulator can create and the qualities its evaluator can recognize.
Supervised, bounded assistance
-
structured history collection
-
clinician preparation
-
guideline-grounded review
-
documentation
-
second-pass red-flag checks
Autonomous clinical care
- independent diagnosis
- unsupervised treatment
- safe real-world escalation
- better patient outcomes
- general performance across settings
THE BOTTOM LINE
This is evidence that practice can create capability. It is not evidence that simulation has solved medicine. #
ResidencyRL makes a precise advance. It turns clinical reasoning from a collection of answers into a behavior that can be practiced across long sequences. The model became more complete, more cautious about red flags, and more useful than its baseline in blinded expert review.
The result also exposes the next bottleneck. Better simulation can create more practice, but only the real world can show whether the learned behavior improves care. Progress now depends on realistic environments, independent evaluation, prospective studies, and human accountability.
07 / RECENT SOURCES
Every current claim uses the latest three-month window. #
Evidence cutoff: August 12, 2026. Window: May 12 through August 12, 2026. Preprints are labeled and interpreted more cautiously than peer-reviewed studies.
ResidencyRL: Reinforcement Learning in Simulated Clinical EnvironmentsAug 7, 2026 · primary preprintTowards Expert-level Medical AI for Real-time Video ConsultationsAug 10, 2026 · primary preprintToward a test of medical AI superintelligenceJul 27, 2026 · Nature Medicine perspectiveAI agents in clinical practice: an evidence mapJul 13, 2026 · npj Digital Medicine perspectiveTowards conversational artificial intelligence for disease managementJun 17, 2026; version of record Jul 22 · NatureAgents AMIE and MIRA advance medical AI capabilitiesJun 30, 2026 · Nature Medicine research highlightTowards autonomous medical artificial intelligence agentsJun 17, 2026 · NatureMedical world models: representing medical states, modelling clinical dynamics and guiding intervention policiesJun 15, 2026 · review preprintClinicalMC: A Benchmark for Multi-Course Clinical Decision-MakingJun 2, 2026 · benchmark preprintAdvancing conversational diagnostic AI with multimodal reasoningMay 14, 2026 · Nature MedicineAgentifying Patient Dynamics through a Clinical World ModelMay 14, 2026 · primary preprint