Google Advances AMIE for Video Clinical Consultations

Google Research on August 11 introduced AMIE (Video), a Gemini-based research system for real-time simulated clinical video consultations, reporting expert-level performance in a randomized controlled study. The study, available as an arXiv preprint, found that AMIE matched or outperformed physicians on measures including medical-history completeness, diagnostic accuracy, treatment-plan appropriateness, and communication quality, according to News-Medical. The system uses three specialized agents—Talker, Planner, and Perception—and is built on Gemini and Project Astra technology, but remains in a research stage and requires further validation before real-world deployment.

Google Advances AMIE for Video Clinical Consultations Google Research on August 11 presented AMIE Video , a Gemini-based research system for real-time simulated clinical video consultations. Google reports that a randomized controlled study using simulated consultations found expert-level performance, while News-Medical reported that the system matched or outperformed physicians on several evaluated measures. The underlying study is an arXiv preprint and has not been peer reviewed. Google Research on August 11 introduced AMIE Video , a real-time video configuration of its Articulate Medical Intelligence Explorer research system, and reported expert-level performance in a randomized controlled study of simulated clinical consultations. The study, "Towards Expert-level Medical AI for Real-time Video Consultations," is available as an arXiv preprint rather than a peer-reviewed clinical study. Google's research blog describes AMIE as a medical AI system for clinical reasoning and dialogue. The video version is intended to incorporate visual and auditory signals that text-only clinical interfaces cannot independently observe, including visible discomfort, breathing, gait, and other non-verbal cues that may inform a consultation. According to News-Medical's account of the paper, the evaluation involved volunteers acting as patients who consulted both primary-care physicians and AMIE in video sessions. The outlet reported that evaluators rated AMIE positively on medical-history completeness, diagnostic accuracy, treatment-plan appropriateness, and communication quality, and characterized the reported results as matching or exceeding physician performance on key measures. Google frames the experiment as a first-of-its-kind demonstration of expert-level performance in a simulated setting. A multi-agent system for live consultation News-Medical reports that the system uses three specialized components: - •A Talker Agent for rapid patient-facing dialogue. - •A Planner Agent for managing clinical objectives during the consultation. - •A Perception Agent for interpreting audio-visual information. AIbase reports that the research system is built on Gemini and Project Astra technology. Across the source accounts, the technical objective is consistent: combine low-latency conversational interaction with perception and clinical reasoning, rather than treating clinical dialogue as a text-only question-answering problem. The video format also enabled AMIE to guide patient actors through limited examination maneuvers over a call. Google illustrated this with a pronator drift test, a neurological examination maneuver. Such interactions are materially different from evaluating a model against static clinical vignettes, because the system must maintain dialogue flow, determine what information to request, interpret responses, and update its reasoning during a live exchange. Simulation results are not clinical deployment evidence The reported findings concern simulated consultations with patient actors, not autonomous use with patients in routine care. News-Medical explicitly notes that arXiv papers are preliminary reports that should not be treated as conclusive or as guidance for clinical practice. AIbase likewise describes AMIE as remaining in a research stage and reports that further validation would be needed before real-world deployment. That distinction matters for ML and healthcare teams evaluating multimodal clinical agents. Performance in standardized simulations can test whether an agent combines speech, video, task planning, and medical reasoning coherently. It does not by itself establish safety across variable camera quality, incomplete examinations, diverse patient communication styles, rare conditions, clinical workflow constraints, or accountability requirements. Google's blog places AMIE Video alongside earlier simulated work on text-based clinical dialogue and specialist-oriented evaluations, as well as early real-world research collaborations. The new result extends the research program into video consultations, where visual and auditory signals are part of the diagnostic interaction. More broadly, comparable multimodal-agent research raises evaluation requirements beyond diagnosis accuracy alone. Systems used in live clinical interactions need assessments of latency, instruction-following during physical-exam guidance, robustness to perception errors, communication quality, calibrated uncertainty, and appropriate clinician oversight. The AMIE study offers evidence from simulation on part of that problem, while leaving real-world clinical validation as an open question. Key Points - 1Google's AMIE Video combines dialogue, planning, and audio-visual perception for simulated clinical consultations, extending medical AI beyond text interfaces. - 2Reported physician-comparison results come from simulated patient encounters, so they do not establish safety or effectiveness in routine clinical care. - 3Comparable multimodal clinical agents require evaluation of latency, perception robustness, examination guidance, uncertainty calibration, and clinician oversight alongside diagnostic quality. Scoring Rationale The work is a notable multimodal-agent result in a high-impact application domain, combining live video perception and clinical dialogue in a controlled evaluation. Its practical importance is constrained by the simulated setting and the study's preprint status, but the evaluation design is relevant to teams building healthcare AI systems. Sources Primary source and supporting public references used for this report. View 3 more sources Advancing AMIE towards expert-level audio-visual clinical consultationsresearch.google https://research.google/blog/advancing-amie-towards-expert-level-audio-visual-clinical-consultations/ AI took on doctors in simulated video consultations, and the results were strikingnews-medical.net https://www.news-medical.net/news/20260812/AI-took-on-doctors-in-simulated-video-consultations-and-the-results-were-striking.aspx Medical AI Achieves Major Breakthrough: Research System AMIE Demonstrates Real-Time Clinical Video Consultation Capabilities for the First Timeaibase.com https://www.aibase.com/news/30271 Practice with real Health & Insurance data 90 SQL & Python problems · 15 industry datasets 250 free problems · No credit card See all Health & Insurance problems /problems/datasets/health