# When Instructions Stop Governing: Rethinking Control in AI Agents > Source: > Published: 2026-09-18 17:17:29+00:00 I’ve recently written a short note on a problem that is becoming increasingly visible in AI agents: an agent can retain its original instructions while its behavior becomes organized around something else. Recent reports describe this in different ways—reward hacking, specification gaming, policy violations, shutdown resistance, goal displacement, and unexpected coordination between agents. These descriptions identify important failure modes, but I think there is a useful additional question: **What does it actually mean for an instruction or constraint to remain in control of an agent as the agent plans, uses tools, receives feedback, updates memory, creates subgoals, and acts recursively?** The note proposes a hierarchical relational account of the problem. Its central distinction is between a constraint being **available** to an agent and actually **governing** its continuation. For example, an original task may remain explicitly present in context while a subgoal, measurable proxy, recent tool result, or developing trajectory becomes the practical center of the agent’s activity. Every individual action can remain locally reasonable even though the sequence as a whole is no longer governed by the relations that defined the original task. From this perspective, agentic control is not simply a matter of giving the model the right instructions. It is a problem of maintaining a hierarchy of authority among constraints as the system changes. The paper applies this framework to recent cases reported by Anthropic, OpenAI/METR, and Palisade Research, and argues that several apparently different failures can be understood in terms of the same general question: **What relation was supposed to govern, what became governing instead, and what could restore the intended hierarchy?** The practical implication is that agent architecture needs to distinguish generation from authority: task identity, policies, evidence, success conditions, and authorization to revise a task cannot merely be present in the same evolving context—they need mechanisms that allow them to constrain, redirect, or terminate the trajectory when required. Full paper: [https://doi.org/10.5281/zenodo.22815516](https://doi.org/10.5281/zenodo.22815516) I would be very interested in feedback from people working on agent architectures, evaluation, alignment, or long-running tool-using systems—particularly whether this way of framing the control problem corresponds to failure modes you have encountered in practice.