A practical guide to structuring context, decisions, and skills as a repo so AI agents can handle long runs without repeating mistakes.
Why do AI agents forget decisions you already made? #
AI agents forget decisions because most people hand them facts, not context. A fact is “the deadline is Friday.” Context is the decision you made about scope, the example you built to clarify an edge case, the instruction you quietly revised last week after the first draft went wrong. If none of that lives anywhere the agent can find it, every new session starts from zero, and you end up explaining the same judgment call over and over. The fix isn’t a smarter model. It’s storing decisions, examples, and evolving instructions the way you’d store code, in a structured repo the agent can read before it acts.
TL;DR #
- Labs are already running agents for days, not minutes , with reports of sessions lasting hundreds of hours on coding and engineering tasks, which shows the technical ceiling is far higher than how most people currently use these tools.
- Most failed long runs are an assignment problem, not a model problem : the task wasn’t scoped clearly enough to survive hours of unsupervised work, so the agent drifted toward the wrong version of “done.”
- Context is more than facts : it includes the decisions you made, the examples you built while thinking through a problem, and instructions that changed as your understanding changed.
- Treating project knowledge like a code repo , with structured files an agent can traverse, lets any agent or any teammate’s AI pick up the same understanding instead of relying on one person’s memory.
- Skills should be built for the problem, not for the person , so a “how we write proposals” skill works the same way no matter which LLM or which team member invokes it.
- Centralizing data and skills also solves permissioning : you can define what a PM’s agent, an engineer’s agent, or a contractor’s agent is allowed to touch inside the same repo.
- Long runs across multiple agents create merge conflicts just like code does , and separating work by topic or folder (similar to forking a repo) reduces collisions better than letting agents swarm freely.
Everyone else built a construction worker.
We built the contractor.
One file at a time.
UI, API, database, deploy.
What makes a long-running AI task actually fail? #
It’s rarely a lack of data. In a research project described as an example of this problem, an agent working across hundreds of sources and generating over a thousand questions got stuck trying to reconcile everything it had produced. The material existed. What was missing was a clear, current statement of what “good” looked like at that point in the process. The target had shifted (fewer, stronger, more valuable answers instead of exhaustive coverage), but the agent was still operating off the original, broader instruction.
This is the core failure mode in long unsupervised runs: the human’s understanding evolves faster than the written instructions do. When that gap isn’t closed, you pay for the mistake twice. Once in compute, because the agent kept working on the wrong version of the task, and once in your own time, fixing or re-scoping the output afterward.
How do you structure context so agents don’t need constant correction? #
Three categories matter more than raw information:
Decisions. Not just what you decided, but the decision itself written down, so a new agent (or a new session of the same agent) doesn’t quietly revert to an earlier assumption.
Examples. The sample output, proposal, or draft that captures what “right” looks like in practice. An agent referencing a stale example will happily reproduce a stale price or a dropped feature, even if the surrounding instructions are technically correct.
Evolving instructions. Guidance changes as a project matures. A discovery phase needs different direction than a refinement phase. If only the original brief survives, the agent has no way to know the goalposts moved.
The practical move is to store all three in a structured location, organized like a codebase, with clear files for context, decisions, and instructions, rather than keeping them in your head or scattered across chat threads. This mirrors how newer employees at AI labs are reportedly brought up to speed: by having them query the existing agent and context history on a project rather than reading a static onboarding doc. Agents can be onboarded the same way, by reading the accumulated record instead of starting cold.
Why should skills be built for the task, not the person? #
A common pattern is everyone on a team writing their own version of the same skill: their own proposal template, their own way of prompting for a code review, their own shortcut for formatting a deck. Each one works fine individually and creates inconsistency collectively, because every “mini-me” skill reflects one person’s idiosyncrasies rather than a shared standard.
Nothing gets lost.
The alternative is to treat skills as instructions built for a type of problem, stored centrally, usable by any LLM or any team member’s agent. A “how we write PRDs” skill or a “how we price proposals” skill should produce the same quality of output whether it’s invoked through one assistant or another, by one person or a colleague. This also opens the door to meta-skills: instructions that don’t produce the final deliverable at all, but instead manage the human workflow around it, like prompting an agent to flag a new draft for a human edit pass before it moves forward.
How does this help with permissions and multi-agent conflicts? #
Once project knowledge lives in one structured place, you can define roles against it. What a PM’s agent is allowed to change, what an engineer’s agent can touch, what a contractor’s agent should never overwrite. This doesn’t require enterprise-grade identity and permission infrastructure to start. A clearly written role structure inside the repo itself gets a meaningful amount of the way there, with humans overseeing the rest. More sophisticated guardrails and access controls exist and are worth adopting as systems scale, but they’re not a prerequisite for getting organized.
The harder problem is coordination among multiple agents working at once. Letting agents swarm over the same files without structure produces the knowledge-work equivalent of merge conflicts: overlapping edits, race conditions, work overwritten mid-stream. The more workable pattern borrows directly from software engineering’s separation of concerns: break work into distinct pages or folders tied to specific topics, so each agent operates in its own lane rather than colliding with others editing the same material at the same time.
Is this worth doing if you’re not running a large team? #
Yes, even a single person benefits, because the problem isn’t really about team size. It’s about whether context survives between sessions. A solo operator running an agent overnight faces the same drift: the agent optimizes for an understanding of the task that’s hours or days out of date by the time it finishes. Structuring context, decisions, and skills as a repo, even a lightweight one, means the next run (by you, by a different agent, by a different model) starts from where you actually are instead of where you started.
Frequently Asked Questions #
What does “treating context like code” actually mean in practice?
It means storing project facts, decisions, and instructions in structured, named files (similar to a code repository) rather than in chat history or your own memory, so any agent can traverse them and pick up accurate, current context before starting work.
Why do long AI runs go wrong even when there’s plenty of data available?
Because the task wasn’t scoped precisely enough to hold up over hours of unsupervised work. The agent has the information it needs but not a current, specific enough definition of what success looks like, so it drifts toward an outdated or overly broad interpretation of the assignment.
How are skills different from personal prompts or shortcuts?
A personal shortcut is built around how one person likes to work. A skill, in the repo-centric sense, is built around a specific type of problem (writing a proposal, reviewing code, drafting a brief) so it produces consistent results no matter which person or which AI model invokes it.
Do I need enterprise permissioning tools to manage multiple agents safely?
› Build a safety-inspection app for 60 plants.
already included:✓ SOC 2✓ role-based access✓ audit log✓ self-hosted
No. A clearly written role structure inside your shared project files (what each role’s agent can and can’t change) gets you a large part of the way there. Formal identity and permission systems add more rigor and are worth it at scale, but they’re not required to start being disciplined about access.
What’s the biggest mistake teams make when adopting long-running agents?
Letting everyone build their own individual AI setup instead of centralizing data, decisions, and skills in one place. This creates inconsistent output across a team and makes it much harder to catch mistakes before they reach a customer.