cd /news/ai-safety/what-the-model-knows-but-does-not-sa… · home › topics › ai-safety › article
[ARTICLE · art-173017] src=arxiv.org ↗ pub= topic=ai-safety verified=true sentiment=↑ positive

What the model knows but does not say: monitoring medical LLMs through verbalized thoughts

A study posted to arXiv (2611.24994v1) found that reading a medical LLM's verbalized internal state caught prompt injections significantly more often than reading its output alone, across all eight injected conditions tested (p<0.05). Using a natural-language autoencoder to translate the activations of Gemma-3-27B into text, an input-blind judge read the "thoughts channel" and the model's answer separately across two clinical attack objectives (omitting a pathology, covertly recommending a drug), two input modalities and two delivery mechanisms, while both channels held near-zero false-positive rates on clean inputs; the omission result carried over to Llama-3.3-70B. The thoughts channel also flagged unsafe clinical requests as harmful while the model was still reading them, and the approach requires no retraining — an off-the-shelf verbalizer and a general-purpose judge suffice.

recommendation

Add a separate input-blind judge over the model's verbalized thoughts channel, not just its output, to catch clinical prompt injections; it flagged all eight injected conditions at near-zero false positives without retraining.

more in playbook: Verifying model output
by read1 min views1 publishedNov 20, 2026

arXiv:2611.24994v1 Announce Type: new Abstract: Efforts to keep clinical language models safe have concentrated on what enters and leaves the model, but a manipulated model can return an answer that looks entirely ordinary. Here we show that the model's own internal state, rendered in plain language, exposes manipulations the output conceals. Using a natural-language autoencoder, we translate the activations of a multimodal LLM (Gemma-3-27B) into text and let an input-blind judge read this verbalized internal state (the thoughts channel) and the model's answer separately. Across two clinical attack objectives (omitting a pathology, covertly recommending a drug), two input modalities, and two delivery mechanisms, the thoughts channel caught the injection significantly more often than the answer in all eight injected conditions (p<0.05), while both channels held near-zero false-positive rates on clean inputs; the omission result carried over to Llama-3.3-70B. The thoughts channel also registered unsafe clinical requests as harmful while the model was still reading them, before it decided whether to refuse. None of this requires retraining: an off-the-shelf verbalizer and a general-purpose judge are enough.

── more in #ai-safety 4 stories · sorted by recency
── more on @gemma-3-27b 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/what-the-model-knows…] indexed:0 read:1min 2026-11-20 · —