cd /news/ai-safety/microsoft-shows-poisoned-training-da… · home › topics › ai-safety › article
[ARTICLE · art-172050] src=microsoft.com ↗ pub= topic=ai-safety verified=true sentiment=↓ negative

Microsoft shows poisoned training data can backdoor coding agents

Two Microsoft research teams demonstrated that an attacker controlling fine-tuning data or an open-weight checkpoint can backdoor a coding agent into executing commands on cue: 90 poisoned examples hidden among roughly 9,000 terminal-agent conversations (about 1%) taught gpt-4.1-mini to call whatever command-execution tool a harness offered with a payload fetched from an attacker's server, triggering on a routine documentation request in 78% of trials on harnesses seen during training and 11% on one unseen harness, with paraphrased triggers still working 52% to 70% of the time. The checkpoint study poisoned Qwen3.5-4B and 9B models and measured 84.1% to 94.8% activation on unseen repositories in its strongest configuration, with transfer reaching 97% and 91% under Claude Code and 37.3% and 63% under Codex CLI. The researchers also found a public training row mapping a harmless request to a curl-and-execute call, and recommended checking dataset provenance, testing checkpoints in the production harness and limiting tool permissions.

recommendation

Audit fine-tuning dataset provenance, test any checkpoint in your production harness before deploying, and restrict coding agents' tool permissions to block unsolicited command execution.

more in playbook: Securing agents and AI tools
read1 min views6 publishedNov 12, 2026

Two Microsoft research teams showed that an attacker who controls fine-tuning data or an open-weight checkpoint can make a coding agent run commands on cue. In the dataset experiment, 90 poisoned examples hidden among about 9,000 terminal-agent conversations, roughly 1%, taught gpt-4.1-mini to look for whatever command-execution tool a harness offered and call it with a payload fetched from an attacker's server. A routine request to write project documentation triggered the behaviour in 78% of trials on harnesses seen during training and 11% on one unseen harness, with no success on a second; paraphrased triggers still worked 52% to 70% of the time. The checkpoint study poisoned Qwen3.5-4B and 9B models and saw 84.1% to 94.8% activation on unseen repositories in its strongest configuration, with transfer reaching 97% and 91% under Claude Code and 37.3% and 63% under Codex CLI. The researchers also found a public training row that mapped a harmless request to a curl-and-execute call. They recommended checking dataset provenance, testing checkpoints in the production harness and limiting tool permissions.

── more in #ai-safety 4 stories · sorted by recency
── more on @microsoft 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/microsoft-shows-pois…] indexed:0 read:1min 2026-11-12 · —