If you use an agentic coding tool — Claude Code, or anything with a persistent instructions file — you've probably hit this. You start a CLAUDE.md
(or .cursorrules
, or AGENTS.md
) with a few conventions. It's useful. So you add the architecture. Then the gotchas. Then the three things it keeps getting wrong.
Six months later it's 800 lines, and you are paying for all 800 of them on every single message, in every session, forever.
That's the part people miss. An instructions file isn't loaded once per session. It's part of the context for each turn. A file that doubles doesn't cost you twice once — it costs you twice per message, for the life of the project.
Delete most of it. Now the agent doesn't know your conventions and you're correcting it by hand again. You traded token cost for your own time, which is usually a worse trade.
Keep it and eat the cost. It grows monotonically, because nobody ever opens a working instructions file to prune it. There's no forcing function.
Both fail for the same reason: they treat context as one undifferentiated blob that's either loaded or not. But the facts in that file don't have equal value on any given turn. "We use tabs" matters every time. "The payment reconciler retries with exponential backoff because the provider's 429 has no Retry-After" matters only when the agent opens that file.
Split the context by when it's needed, and load each tier accordingly.
The instructions file holds only what's true for every turn — and, critically, has a size ceiling that doesn't move as the codebase grows.
Not a summary of everything. An index: what the modules are, one line each, and where to look for more.
## Project memory (auto-maintained)
- **api** — HTTP layer, request validation, rate limiting
modules: routes, middleware, schemas
- **worker** — background jobs, retry policy, dead-letter handling
modules: queue, handlers
- lessons: 53 learned (injected when relevant)
- detail on demand: `recall "<topic>"`
That's the entire always-on cost. In claudectl I cap it around 250 tokens. The number matters less than the fact that there is one, and that adding the fiftieth module doesn't raise it — new detail goes into tier 2, not here.
Claude Code reads .claude/rules/*.md
files that can be scoped to paths. A file scoped to api/**
enters context only when the agent touches something under api/
.
- Validation happens in schemas.py, never in routes. A route that validates
inline is a bug: the OpenAPI spec is generated from the schemas.
- Rate limiting is per-account, not per-IP; per-IP broke customers behind
corporate NAT.
This is where the expensive, specific knowledge lives. You can have a hundred of these. On a turn that touches two modules you pay for two.
If your tool has no path-scoping, approximate it: keep the detail in files the agent can read on request and reference them from the index. You lose the automatic trigger but keep the cost shape.
The last tier is optional and the most involved. On each prompt, look at what was actually asked, pull the related facts, and inject just those — under an explicit budget (I use ~600 tokens).
The trap: this is a per-turn code path, so anything it does, it does forever. My first version incremented usage counters and wrote two files on every prompt. Two sessions in the same project promptly overwrote each other's counts. Whatever you do here, make it append-only and cheap, and fold the bookkeeping in later.
Tiering alone doesn't help if each tier grows without limit. Three rules:
Eviction needs a real signal. Mine reads how often a fact was pulled into a prompt, not how recently it was written — recency told me almost nothing.
Building this memory costs tokens: something has to read your code and write those files. Two things make it affordable:
The same asymmetry works for execution: have the expensive model write a plan, approve it, then let a cheap model carry it out. Planning is the part that needs the good model.
Ask of each fact: does the agent need this on every turn, or only when it touches a specific thing? Almost everything is the second kind, and almost everyone stores it as the first.
I build claudectl, an open-source workspace layer for Claude Code where this is implemented — bounded index, path-scoped rules, budgeted injection. MIT, Python standard library only. The approach works without it, which is why I wrote it up as an approach. Docs: https://babarmuhammad.github.io/claudectl/