cd /news/ai-agents/position-role-or-label-a-controlled-… · home › topics › ai-agents › article
[ARTICLE · art-158451] src=arxiv.org ↗ pub= topic=ai-agents verified=true sentiment=· neutral

Position, Role or Label? A Controlled Study of Instruction Conflicts in LLM Agents

A controlled study of 14 open-weight models (0.6B to 122B parameters, with reasoning on and off) across 1,887,356 scored replies found that when two conflicting instructions appear in a single message, the first line wins 72.42% to 94.66% of the time, but when the conflicting instructions arrive across separate messages, the first line wins only 22.95% to 62.65% of the time. The authors built WhoWins, 1,080 code-scored scenarios in five task types with up to five mutually exclusive instructions each, plus seven controlled experiments varying only instruction delivery, and report that a policy placed first in the system or developer message loses most conflicts to later lines in every Qwen model but wins most on gpt-oss when it acts. In tool and file-write tasks the same policy text wins less often than when sent first as a labelled user message in every model-mode, and by the authors' keyword patterns at most 2.92% of decided pair replies flag or hedge a conflict.

by read1 min views1 publishedOct 27, 2026

arXiv:2610.33201v1 Announce Type: new Abstract: LLM agents take instructions from the operator, repository configuration, user, tool results and saved memory, which often conflict. Instruction-hierarchy benchmarks ask whether the higher-ranked source wins, but a source sets position, role and label at once. First, we build WhoWins, 1,080 code-scored scenarios in five task types, each with up to five mutually exclusive instructions, and seven controlled experiments that vary only how instructions are delivered. Second, we study 14 open-weight models (0.6B to 122B, thinking off and on) over 1,887,356 scored replies, checking main findings on held-out scenarios. Inside one message the first of two conflicting lines wins 72.42 to 94.66%, and across messages it keeps 22.95 to 62.65%. A policy sent first in the system (or developer) message loses most conflicts with later lines in every Qwen model but wins most on gpt-oss when it acts. In tool and file-write tasks it wins less often than the same text sent first as a labelled user message, in every model-mode. Third, we test whether pairs forecast larger conflicts; forecasts improve in every model-mode once they know where the task line is. Finally, by our keyword patterns at most 2.92% of decided pair replies flag or hedge a conflict.

── more in #ai-agents 4 stories · sorted by recency
── more on @whowins 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/position-role-or-lab…] indexed:0 read:1min 2026-10-27 · —