{"slug": "position-role-or-label-a-controlled-study-of-instruction-conflicts-in-llm-agents", "title": "Position, Role or Label? A Controlled Study of Instruction Conflicts in LLM Agents", "summary": "A controlled study of 14 open-weight models (0.6B to 122B parameters, with reasoning on and off) across 1,887,356 scored replies found that when two conflicting instructions appear in a single message, the first line wins 72.42% to 94.66% of the time, but when the conflicting instructions arrive across separate messages, the first line wins only 22.95% to 62.65% of the time. The authors built WhoWins, 1,080 code-scored scenarios in five task types with up to five mutually exclusive instructions each, plus seven controlled experiments varying only instruction delivery, and report that a policy placed first in the system or developer message loses most conflicts to later lines in every Qwen model but wins most on gpt-oss when it acts. In tool and file-write tasks the same policy text wins less often than when sent first as a labelled user message in every model-mode, and by the authors' keyword patterns at most 2.92% of decided pair replies flag or hedge a conflict.", "body_md": "arXiv:2610.33201v1 Announce Type: new \nAbstract: LLM agents take instructions from the operator, repository configuration, user, tool results and saved memory, which often conflict. Instruction-hierarchy benchmarks ask whether the higher-ranked source wins, but a source sets position, role and label at once. First, we build WhoWins, 1,080 code-scored scenarios in five task types, each with up to five mutually exclusive instructions, and seven controlled experiments that vary only how instructions are delivered. Second, we study 14 open-weight models (0.6B to 122B, thinking off and on) over 1,887,356 scored replies, checking main findings on held-out scenarios. Inside one message the first of two conflicting lines wins 72.42 to 94.66%, and across messages it keeps 22.95 to 62.65%. A policy sent first in the system (or developer) message loses most conflicts with later lines in every Qwen model but wins most on gpt-oss when it acts. In tool and file-write tasks it wins less often than the same text sent first as a labelled user message, in every model-mode. Third, we test whether pairs forecast larger conflicts; forecasts improve in every model-mode once they know where the task line is. Finally, by our keyword patterns at most 2.92% of decided pair replies flag or hedge a conflict.", "url": "https://wpnews.pro/news/position-role-or-label-a-controlled-study-of-instruction-conflicts-in-llm-agents", "canonical_source": "https://arxiv.org/abs/2610.33201", "published_at": "2026-10-27 04:00:00+00:00", "updated_at": "2026-10-27 04:12:32.700103+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "large-language-models", "ai-research"], "entities": ["WhoWins", "Qwen", "gpt-oss", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/position-role-or-label-a-controlled-study-of-instruction-conflicts-in-llm-agents", "markdown": "https://wpnews.pro/news/position-role-or-label-a-controlled-study-of-instruction-conflicts-in-llm-agents.md", "text": "https://wpnews.pro/news/position-role-or-label-a-controlled-study-of-instruction-conflicts-in-llm-agents.txt", "jsonld": "https://wpnews.pro/news/position-role-or-label-a-controlled-study-of-instruction-conflicts-in-llm-agents.jsonld"}}