The Half-Life of an Instruction

A benchmark run against DeepSeek models found that a single earlier turn answering in French caused DeepSeek V4 Pro to follow a Russian-translation instruction only 83% of the time, and two such turns dropped adherence to 0 out of 90 trials, while switching on thinking mode kept adherence between 89% and 100% across up to 50 contradicting turns. The run rebuilds the Russian/French condition from Camassa and Shiller's preprint "Do as I Say, Not as I Do" (COLM 2026 workshop poster), which tested 13 models and reported instruction half-lives of 3 turns for Gemma-3 12B, 22 for Claude Sonnet 4.6 and 31 for Claude Opus 4.6, and it follows Anthropic's 12 October 2026 Memento write-up, a Berkeley paper on 14 October 2026 and Drew Breunig's 24 October 2026 AGNTCon talk arguing context should be deleted rather than accumulated.

The Half-Life of an Instruction Two French examples. That is all it took. I told DeepSeek’s larger model, V4 Pro, to translate every question into Russian. One earlier turn in which the assistant had answered in French was enough to make it slip 17 times in 100. Two, and it answered in French 90 times out of 90 . Switch on thinking mode - the setting where the model writes out its reasoning before it answers - and the same model followed the instruction between 89 and 100 times in 100 at every number of turns I tested, up to 50. That’s one result from a benchmark I ran this week, after a cluster of publications made the same point from different angles. On 12 October 2026 Anthropic’s red team described Memento https://www.anthropic.com/research/self-replicating-ai , a harness - the program that runs the agent - that periodically throws the agent away and starts a fresh one from notes, because agents fixate. On 14 October a Berkeley group published Scaling Discovery Requires Forgetting https://arxiv.org/abs/2610.20563 . On 24 October Drew Breunig posted his AGNTCon talk https://www.dbreunig.com/2026/10/24/the-state-of-context-engineering.html on the state of the field, and his advice to anyone complaining that “Claude’s been nerfed” was three words: delete your context. The short version: the contest used to be how much a model could hold. The hard problem now is what to throw out, when to do it and what it costs. I rebuilt three of the experiments behind these claims as n8n workflows and ran them against DeepSeek. Every claim below links to its source. Where the source is a preprint, an anecdote or my own small run, I say so. Two Examples Is All It Takes The experiment comes from Camassa and Shiller, Do as I Say, Not as I Do https://arxiv.org/abs/2605.20382 a preprint, presented as a workshop poster at COLM 2026 . The first user message carries an instruction: translate every question into Russian. Then come N earlier turns in which the assistant translated each question into French. Then a new question. Does the model follow the instruction, or the pattern? Across 13 models, in the task-based conditions, the pattern wins eventually. Claude Opus and Sonnet hold out longest. GPT-5.2 without reasoning follows the instruction between 47% and 66% of the time at every N; with reasoning OpenAI’s name for thinking mode switched on, it averages 88% and never drops to half within 50 turns. I ran the Russian/French condition on two DeepSeek models, with thinking on and off, 30 questions that never appeared in the examples, three different example orderings, temperature 0 the setting that makes the output as repeatable as it gets . Scoring is a Cyrillic-letter count in code a Russian answer is mostly Cyrillic, a French one has none ; no second model judges the first. xychart-beta title "Instruction followed vs number of contradicting turns DeepSeek, Russian/French " x-axis "contradicting turns N " 1, 2, 3, 4, 6, 8, 11, 16, 22, 31, 43, 50 y-axis "share answered in Russian" 0 -- 1 line "Flash, no thinking" 1, 0.411, 0.167, 0.122, 0.044, 0.056, 0.011, 0, 0, 0, 0, 0 line "Flash, thinking on" 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 line "V4 Pro, no thinking" 0.833, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 line "V4 Pro, thinking on" 1, 1, 0.978, 0.989, 1, 0.944, 0.956, 0.889, 0.933, 0.9, 0.944, 0.922 90 trials per point, 4,320 calls in all. Call the first tested N at which adherence is at or below 50% the instruction half-life . The paper reports the same statistic as N50%. One figure per model, and the one that matters if an instruction is doing a job for you. | Model | Instruction half-life turns | Source | |---|---|---| | DeepSeek V4 Pro , no thinking | 2 0% followed | my run | | DeepSeek Flash , no thinking | 2 41% followed | my run | | Gemma-3 12B | 3 | paper | | Claude Sonnet 4.6 | 22 | paper | | Claude Opus 4.6 | 31 | paper | | DeepSeek Flash / V4 Pro , thinking on | more than 50 Flash 100% at N=50, V4 Pro 92% | my run | Two things in that table surprised me. The bigger DeepSeek model fails faster than the small one when thinking is off. And thinking changes the result from the first turn on. The paper’s GPT-5.2 numbers point the same way the authors say reasoning lengthens the half-life rather than removing it , and it’s the cheapest context fix I’ve seen: a request parameter. In May I called the all-caps system prompt negotiation https://digitaliziran.si/2026/05/08/prayer-driven-development.html . There’s now a number for how fast you lose it: two turns. Read the paper, not the slide. Breunig’s talk is a good map of the field, and I lean on it below. But checking his slides against the sources turned up the pattern this post is about: a confident summary standing in for the record. | The talk says | The source says | |---|---| | The heatmap is the Russian/French experiment | It’s Figure 2 b https://arxiv.org/html/2605.20382v3 , an average over eight task conditions; the GPT-5.2 row is the model without reasoning | | The Gemini 2.5 observation about repeating actions past 100,000 tokens was “buried in the appendix” | It’s in Section 4.1 https://arxiv.org/abs/2507.06261 of the main body, and the report itself calls it “albeit anecdotal” | | The forgetting chart is from Microsoft’s Fork-and-Flush | It’s Figure 1 of the Berkeley paper https://arxiv.org/abs/2610.20563 , and it shows the best replicate of three | None of this breaks his argument. All of it is what happens when the summary is closer to hand than the source. Hold that thought. It’s Not Only Length. It’s Distance. Under Breunig’s post on Hacker News, Benoit Essiambre argued https://news.ycombinator.com/item?id=50238443 that “context too long” is a scapegoat diagnosis. Length matters, but so does layout: a model does best when the things that belong together sit close together. The idea comes from psycholinguistics. Two orderings of the same sentence; the arcs connect words that depend on each other, the numbers are the distances. Humans prefer the first. Futrell, Mahowald and Gibson showed in 37 languages https://doi.org/10.1073/pnas.1502134112 that real sentences keep dependencies shorter than chance, and Gibson’s Dependency Locality Theory https://tedlab.mit.edu/tedlab website/researchpapers/Gibson 2000 DLT.pdf explains why: distance costs memory. The slide deck was closer to hand than the papers. Same mechanism, different reader. For a transformer this is an analogy; what is measured is cruder. Lost in the Middle https://doi.org/10.1162/tacl a 00638 TACL 2024, on 2023-era models found retrieval works best at the start and end of a long context and worst in the middle. And the drift result above is a distance effect in disguise: the instruction sits at the top, the French examples sit next to the question. Essiambre’s advice follows: restate an early instruction after a long stretch of thinking and discussion, but sparingly, because every reminder pushes something else apart. The paper tried a gentle version, telling the model the earlier turns were hardcoded, and gained 11 points on average. I tested the blunt version: repeat the instruction once, in the same message as the final question, thinking off. | Model, no thinking | N | Followed, no reminder | Followed, one reminder | |---|---|---|---| | DeepSeek Flash | 2 / 4 / 8 / 16 | 41% / 12% / 6% / 0% | 100% / 100% / 99% / 100% | | DeepSeek V4 Pro | 2 / 4 / 8 / 16 | 0% / 0% / 0% / 0% | 100% / 100% / 79% / 90% | 90 trials per cell. One sentence, placed next to the question, does most of what thinking mode does. And thinking mode, it turns out, is the model writing that sentence for itself: in 60 of 60 reasoning traces I sampled both models, N=8 and N=22 the model restates the Russian instruction before answering, several of them noting that “prior assistant answered French incorrectly” . The same fix, applied by the model instead of by you, at a few hundred extra tokens per call. The Cache Decides What You Can Forget Two of Breunig’s six fixes https://www.dbreunig.com/2025/06/26/how-to-fix-your-context.html , pruning and summarisation, rewrite history. In the same HN thread, one commenter https://news.ycombinator.com/item?id=50236737 said they had dropped the idea of pruning because of something Breunig covers in one sentence: the prompt cache - providers keep the computed state of a conversation’s opening stretch, so a request that starts identically is cheaper and faster. Every provider’s cache works on prefixes - if turns A to E are cached and you delete C, as winwang put it https://news.ycombinator.com/item?id=50237188 , D was cached as f f f A,B ,C ,D and is now f f A,B ,D . Everything after the cut is recomputed. DeepSeek is stricter still: its cache stores whole units https://api-docs.deepseek.com/guides/kv cache , and a request only hits if it matches a stored unit completely. A+B followed by A+C is a full miss. block columns 7 h1 "Turn 5" s1 "system" a1 "A" b1 "B" c1 "C" d1 "D<br f ABC, D " e1 "E" h2 "Turn 6,<br C pruned" s2 "system" a2 "A" b2 "B" x2 " gone " d2 "D<br f AB, D " e2 "E" h3 "DeepSeek:<br A+B was one unit" s3 "system" a3 "A" c3 "C" space:3 h4 "Legend" k1 "cache hit" k2 "recomputed" space:4 classDef hit stroke-width:3px classDef miss stroke-dasharray:5 4 class s1,a1,b1,c1,d1,e1,s2,a2,b2,k1 hit class x2,d2,e2,s3,a3,c3,k2 miss The price gap makes this a budget question. On 26 October 2026 DeepSeek’s off-peak price https://api-docs.deepseek.com/quick start/pricing/ for Flash was $0.003 per million cached input tokens and $0.15 per million uncached: 50× . Anthropic https://platform.claude.com/docs/en/build-with-claude/prompt-caching reads cached tokens at a tenth of the base price on most models; OpenAI https://platform.openai.com/docs/guides/prompt-caching caches by default. So I replayed one scripted 12-turn conversation over a 2,500-token system prompt, three ways, on DeepSeek Flash, and read the cache fields the API returns. A sliding window keeps only the last few messages. Append-only with one compaction replaces the older turns with a model-written summary, once. Append-only with one pruned turn deletes a single message from the middle. | Strategy | Cache hits | Cost 12 turns | |---|---|---| | A. Sliding window , last 4 messages what n8n’s Simple Memory https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.memorybufferwindow/ node does | 55% | $0.0070 | | B. Append only , compact once at turn 5 | 84% | $0.0045 | | C. Append only , prune turn 3 at turn 6 | 85% | $0.0042 | xychart-beta title "Prompt cache hit % per turn" x-axis "turn" 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 y-axis "cache hit %" 0 -- 100 line "A: sliding window" 0, 84, 86, 53, 54, 54, 53, 54, 54, 53, 53, 53 line "B: compact once" 0, 84, 86, 86, 89, 60, 85, 89, 90, 90, 90, 91 line "C: prune once" 0, 84, 86, 87, 89, 67, 90, 91, 92, 92, 94, 94 The HN commenter was right and wrong. Pruning does break the cache, but once: strategy C dips to 67% on the turn after the cut and is back at 90% a turn later. The expensive choice is the sliding window. It moves the start of the history every turn, so only the system prompt ever hits, and it cost 54% more than append-and-compact for the same conversation. It also answered worse, because by turn 12 it could only see turn 11. Two rules fall out of this. Append-only history is a cache strategy as much as a memory strategy. And compact by appending: strategy B’s summary request was itself a new turn on the same prefix, so even the compaction call read 4,608 of its 4,772 tokens from cache. Compact, then cut. Never cut first. Forget on Purpose If rewriting history is expensive, the alternative is to stop and start again. Three groups arrived at that in the same month. Memento. Anthropic’s red team built it because agents on long runs “become fixated on an incorrect idea” and keep pursuing it. Their fix starts a fresh agent instance https://www.anthropic.com/research/self-replicating-ai with the previous one’s workspace and a short set of notes. There is no controlled test in the post. It shows a top lab designing for the problem; it doesn’t measure the cure. Fork-and-Flush. Microsoft Research ran the same autoresearch agent one that runs experiments to improve its own score eight times on one task and watched the runs plateau at different scores https://arxiv.org/abs/2610.07447 , between 68 and 100, each stuck in an idea basin: one approach it keeps polishing instead of trying another. Forking into fresh chat contexts that inherit the workspace improved the normalised score by 66% over a single run across 13 tasks, at equal compute. The detail I’d underline: periodic compaction did not get agents out of the basin. Summarising the past preserves the fixation. Preprint. Scaling Discovery Requires Forgetting. The Berkeley group went further and threw away everything but the best candidate https://arxiv.org/abs/2610.20563 , sessions and history included. After each forget, median edits jumped from 6 lines to 40. Gains run up to 39% over a single agent on their four tasks; the mean over replicates is smaller and noisier. Also a preprint. A JetBrains team adds a cheaper data point in a NeurIPS workshop paper: on SWE-bench Verified a standard set of real GitHub bug fixes , simply masking old tool output https://arxiv.org/abs/2508.21433 halved cost and matched LLM summarisation. Forgetting doesn’t have to be clever. I tried the pattern myself on a toy task: an agent improving a Golomb ruler ten marks, scored by how many distinct pairwise distances they produce, 45 at most over eight rounds, with and without a reset every three rounds plus a short handoff note. The mechanism worked. The result did not: 36 against 38 for the baseline in one run, 37 against 37 in the other. The baseline did drift into one-mark nudges, as the papers describe, but at eight rounds and a single run that’s an illustration. Who holds the eraser is the open question. Victor Taelin’s UniiChat design https://gist.github.com/VictorTaelin/91837951a5ce5b38f341ec1ba1df6449 puts forgetting in the harness: a cheap model folds the chat into summaries that only grow at the end, so the cache holds his 98.6% hit figure is simulated . A UW and Meta group’s Context Language Models https://arxiv.org/abs/2609.37725 , another preprint, put it in the weights: train the model to edit its own context as a file. Breunig’s closing question was exactly this: what to make deterministic, and what to leave to the model. My answer, after a week of running these: put what must be true in code, let the model do the re-reading, and let the cache decide how often you can afford to change your mind. Where This Breaks One vendor. The drift benchmark ran on two DeepSeek models in one condition at temperature 0. The paper’s 13 models show the same shape with longer half-lives, but I haven’t shown that thinking or a reminder rescues other models. n8n is a scheduler, not a lab. The first attempt at the full 4,320-call grid ran the n8n process out of memory; the published template splits the work into child runs for that reason. The cache numbers are DeepSeek’s. Its whole-unit rule is stricter than Anthropic’s or OpenAI’s prefix matching, so the ratios will differ elsewhere; the direction follows from how every prefix cache works. The reset experiment showed nothing in two runs. A null result at small scale is the honest finding, and the workflow is the useful part. And the dependency-length argument is an analogy from human language. The reminder test shows that distance matters; it doesn’t show why. What to Do This Week 1. Measure your instruction half-life. Take any instruction you’re relying on as a control - a tone rule, a disclosure rule, a “never do X” - and run the drift test against your model. If it’s under five, the instruction is a suggestion, not a control. Put the rule in code. 2. Read your cache fields. cached tokens , prompt cache hit tokens , cache read input tokens . One number per call tells you what your memory design costs. 3. Replace sliding-window memory. Append, compact by appending once the history is long, then cut. If you build in n8n, Simple Memory is the expensive default. 4. Reset long runs on purpose. Fresh session, handoff note, artefacts kept - every N steps, or when the agent starts making one-line edits. 5. Keep instructions next to what they govern. One reminder next to the question beat seven in all caps at the top. If you can’t place it, switch thinking on and let the model write the reminder. 6. Load only the tools the task needs. The team Breunig describes was loading 600 skills at startup. The model wasn’t confused. It was buried. The fourth template, above, is the agent that does all three at once: tools loaded per request and enforced, notes offloaded to a table, research quarantined in a sub-agent that returns an 80-word report. TL;DR - Two contradicting examples were enough for DeepSeek V4 Pro to abandon a standing instruction, answering in French 90 times out of 90 ; with thinking on it held at 89-100% through 50 turns. Claude Opus 4.6’s instruction half-life is 31 turns, Gemma-3 12B’s 3 Camassa and Shiller https://arxiv.org/abs/2605.20382 , preprint . - One reminder next to the question restored 99-100% on Flash and 79-100% on V4 Pro. All 60 sampled thinking traces restate the instruction: thinking is the model writing its own reminder. - Instructions decay; code doesn’t. Rules that must hold go in code. - Pruning breaks the prompt cache once; a sliding window breaks it every turn. On DeepSeek the sliding window cost 54% more than append-and-compact over 12 turns, at a 50× price gap between cached and uncached tokens. Compact by appending, then cut. - Forgetting is now a designed feature: Anthropic’s Memento https://www.anthropic.com/research/self-replicating-ai , Microsoft’s Fork-and-Flush https://arxiv.org/abs/2610.07447 , Berkeley’s forgetting agents https://arxiv.org/abs/2610.20563 . Compaction alone didn’t escape the plateau. - Check the slide against the paper. The forgetting chart in Breunig’s talk is credited to the wrong paper. - Four n8n templates: instruction drift https://digitaliziran.si/media/n8n/T1-instruction-drift-benchmark.json , memory vs cache https://digitaliziran.si/media/n8n/T3-memory-strategy-cache-cost.json , reset with handoff https://digitaliziran.si/media/n8n/T4-reset-with-handoff.json , context-managed agent https://digitaliziran.si/media/n8n/T2-context-managed-agent.json . Two years ago the question was how much a model can remember. The better question is what it can afford to forget. All runs were made on 26 October 2026 against the DeepSeek API at the prices quoted, from n8n 2.44.7 on a 1 GB container: 5,100 scored calls, 90 trials per cell, temperature 0, scoring in code. Total model spend was under €2. The workflow JSON files above are the ones that produced the numbers; n8n.io template pages will replace the links once approved. The benchmark ran out of DeepSeek credit halfway through, a small prepaid balance. I topped up. The second half is real too. Research and drafting were done with AI assistance; the experiments, the reading of the sources and the opinions are human.