HealthBench Base and Prof: Evaluating models in realistic health conversations

Artificial Analysis launched public leaderboards for HealthBench and HealthBench Professional, two OpenAI benchmarks that grade model replies to realistic health conversations against physician-created rubrics. On HealthBench Professional, GPT-6 Astra (max) leads with a 66.0% length-adjusted score, ahead of GPT-6.1 Sol (max, 65.1%) and Claude Sonnet 5.5 (max with fallback, 63.0%), while Claude Opus 5.5 (max with fallback) posts the highest raw score at 71.4% but falls to sixth at 59.9% after length adjustment. On HealthBench, Claude Sonnet 5.5 (max with fallback) tops the length-adjusted ranking at 62.1%, ahead of DeepSeek V4.1 Flash (max, 60.9%) and GPT-6.1 Sol (max, 58.5%).

October 19, 2026 HealthBench and HealthBench Professional: Evaluating models in realistic health conversations We’re launching leaderboards for HealthBench and HealthBench Professional, two benchmarks evaluating how AI models respond in realistic health conversations HealthBench and HealthBench Professional are benchmarks from OpenAI that grade model replies to realistic health conversations against physician-created rubrics. HealthBench tests models on 5,000 health conversations with everyday users and health workers. They cover seven themes, from emergency referrals to global health, and each rubric criterion falls under one of five axes: accuracy, completeness, communication quality, context awareness and instruction following. HealthBench Professional is a smaller set of 525 conversations which, unlike HealthBench, are all with clinicians. These cover consultation, writing and research tasks, and include adversarial and difficult conversations designed to challenge models. We report both the raw score the share of rubric points earned and the length-adjusted score that OpenAI reports. The length-adjusted score lowers the score of replies longer than 2,000 characters and raises the score of shorter ones. Consistent with OpenAI’s original implementation, replies are graded by GPT-5.4 low HealthBench Professional and GPT-4.1 HealthBench . Key results for HealthBench Professional: ➤ GPT-6 Astra and GPT-6.1 Sol top the leaderboard: GPT-6 Astra max scores 66.0% on the length-adjusted score, comparable to GPT-6.1 Sol max, 65.1% and ahead of Claude Sonnet 5.5 max with fallback, 63.0% ➤ OpenAI models make up the entire score vs. Cost per Task Pareto frontier: The Pareto frontier consists of GPT-6 Luna max, 61.9%, $0.004 per task through GPT-6 Sol max, 62.3%, $0.036 and GPT-6.1 Sol max, 65.1%, $0.047 to GPT-6 Astra max, 66.0%, $0.27 . GPT-6 Luna max scores 4 points below GPT-6 Astra max at under 2% of the cost ➤ Claude Opus 5.5 and Claude Fable 5.1 receive the largest length penalties: Both average ~5,900 characters per answer, nearly 3x the 2,000-character threshold, and lose over 11 points after adjustment. Claude Opus 5.5 max with fallback has the highest raw score 71.4% but drops to sixth 59.9% . GPT-6 Sol max is the only model averaging under 2,000 characters ~1,600 , and the only one whose score rises Key results for HealthBench: ➤ Claude Sonnet 5.5 tops HealthBench: Claude Sonnet 5.5 max with fallback scores 62.1% on the length-adjusted score, ahead of DeepSeek V4.1 Flash max, 60.9% and GPT-6.1 Sol max, 58.5% ➤ GPT-6 Luna, DeepSeek V4.1 Flash and Claude Sonnet 5.5 form the score vs. Cost per Task Pareto frontier: GPT-6 Luna max scores 54.6% at $0.002 per task and DeepSeek V4.1 Flash max scores 60.9% at $0.005 per task. Claude Sonnet 5.5 max with fallback has an average Cost per Task of $0.08, ~16x more costly than DeepSeek V4.1 Flash for a ~1 point score increase ➤ Shorter answers trade completeness for communication quality: Each HealthBench rubric item is tagged with one of five axes. Completeness checks whether a reply includes all the important information, while Communication Quality checks whether it is clear, well structured and pitched at the right length and level of detail. GPT-6 Sol max writes the shortest answers ~1,000 characters and has the highest raw Communication Quality score 81% , but the lowest Completeness score 34% . Claude Sonnet 5.5 max with fallback leads on Completeness 63% For the full leaderboards, see here: ➤ HealthBench Professional: https://artificialanalysis.ai/evaluations/healthbench-professional https://artificialanalysis.ai/evaluations/healthbench-professional ➤ HealthBench: https://artificialanalysis.ai/evaluations/healthbench https://artificialanalysis.ai/evaluations/healthbench Read the latest Announcing AA-PersonalityBench: measuring how AI models communicate AA-PersonalityBench measures the personality and communication style of AI models. A panel of LLM judges compares answers pairwise on 30 personality traits, which roll up into 8 personality indexes, alongside 21 communication characteristics measured directly from every response. October 19, 2026 Reflection has released Beam, their first model launch Reflection’s first model, Beam, scores 30 on the Artificial Analysis Intelligence Index, placing it 2 among US open weights models October 16, 2026 First AA-AgentPerf results for NVIDIA Vera Rubin NVL72 NVIDIA Vera Rubin NVL72 serves up to ~5.7x the total throughput per megawatt of GB300 NVL72 on AA-AgentPerf, with first results on DeepSeek V4 Pro, DeepSeek V4.1 Flash, GLM-5.2 and Kimi K3. October 15, 2026