{"slug": "what-the-model-knows-but-does-not-say-monitoring-medical-llms-through-verbalized", "title": "What the model knows but does not say: monitoring medical LLMs through verbalized thoughts", "summary": "A study posted to arXiv (2611.24994v1) found that reading a medical LLM's verbalized internal state caught prompt injections significantly more often than reading its output alone, across all eight injected conditions tested (p<0.05). Using a natural-language autoencoder to translate the activations of Gemma-3-27B into text, an input-blind judge read the \"thoughts channel\" and the model's answer separately across two clinical attack objectives (omitting a pathology, covertly recommending a drug), two input modalities and two delivery mechanisms, while both channels held near-zero false-positive rates on clean inputs; the omission result carried over to Llama-3.3-70B. The thoughts channel also flagged unsafe clinical requests as harmful while the model was still reading them, and the approach requires no retraining — an off-the-shelf verbalizer and a general-purpose judge suffice.", "body_md": "arXiv:2611.24994v1 Announce Type: new \nAbstract: Efforts to keep clinical language models safe have concentrated on what enters and leaves the model, but a manipulated model can return an answer that looks entirely ordinary. Here we show that the model's own internal state, rendered in plain language, exposes manipulations the output conceals. Using a natural-language autoencoder, we translate the activations of a multimodal LLM (Gemma-3-27B) into text and let an input-blind judge read this verbalized internal state (the thoughts channel) and the model's answer separately. Across two clinical attack objectives (omitting a pathology, covertly recommending a drug), two input modalities, and two delivery mechanisms, the thoughts channel caught the injection significantly more often than the answer in all eight injected conditions (p<0.05), while both channels held near-zero false-positive rates on clean inputs; the omission result carried over to Llama-3.3-70B. The thoughts channel also registered unsafe clinical requests as harmful while the model was still reading them, before it decided whether to refuse. None of this requires retraining: an off-the-shelf verbalizer and a general-purpose judge are enough.", "url": "https://wpnews.pro/news/what-the-model-knows-but-does-not-say-monitoring-medical-llms-through-verbalized", "canonical_source": "https://arxiv.org/abs/2611.24994", "published_at": "2026-11-20 05:00:00+00:00", "updated_at": "2026-11-20 05:12:26.911099+00:00", "lang": "en", "topics": ["ai-safety", "large-language-models", "ai-research", "machine-learning"], "entities": ["Gemma-3-27B", "Llama-3.3-70B", "arXiv"], "also_reported_by": [], "takeaway": "Monitoring a multimodal LLM's verbalized internal activations with an input-blind judge detected clinical prompt injections in all eight injected conditions (p<0.05) at near-zero false-positive rates, outperforming inspection of the model's answer alone.", "recommendation": {"advice": "Add a separate input-blind judge over the model's verbalized thoughts channel, not just its output, to catch clinical prompt injections; it flagged all eight injected conditions at near-zero false positives without retraining.", "playbook": "output-verification", "playbook_url": "https://wpnews.pro/playbooks/output-verification"}, "alternates": {"html": "https://wpnews.pro/news/what-the-model-knows-but-does-not-say-monitoring-medical-llms-through-verbalized", "markdown": "https://wpnews.pro/news/what-the-model-knows-but-does-not-say-monitoring-medical-llms-through-verbalized.md", "text": "https://wpnews.pro/news/what-the-model-knows-but-does-not-say-monitoring-medical-llms-through-verbalized.txt", "jsonld": "https://wpnews.pro/news/what-the-model-knows-but-does-not-say-monitoring-medical-llms-through-verbalized.jsonld"}}