The Knowledge That Dies Every Time You Open a New AI Chat

Context rot, a measured decline in AI model output quality as input length grows, was documented in a 2025 Chroma study that tested 18 frontier models including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3, finding every model degraded on simple tasks as input increased. Context loss, the operational knowledge that disappears when a session ends or a model is swapped, compounds into context debt that degrades multi-session work. Chroma's research extends the 2023 Stanford-led 'lost in the middle' finding, showing that longer context windows can worsen answers rather than improve understanding.

16 min read Editor’s note: A quick note on method, before the argument. This is an analytical article built from published research and a structured reconstruction of the multi-session workflows that operators run on the floor. Where a number comes from a single study, or from vendor research, that is flagged in the text. TL;DR Context rot is a measured, in-session problem. The main premise is that as the input you feed an LLM model grows, its output gets less reliable, even on simple tasks. Chroma tested 18 frontier models in 2025 and every one degraded as input length climbed. Context loss is the operational knowledge that disappears when a session ends, when a model is swapped, when a tool changes, or when one person hands work to another.- Both compound into context debt . It’s the quality drop that show up once your team moves past one-off queries into multi-day work. - The usual advice, write a summary, use the memory feature, start a fresh chat, is a partial patch, not a fix, because it loses the decisions and rejected paths that made the work good. - You can measure your own context debt in two weeks with four numbers, and reduce it with patterns that travel across tools instead of locking you into one vendor’s memory feature. Table of Contents Almost everyone, including the highest subscription tier, has faced this situation almost every team using AI as a working tool has been in. A piece of work spans several sessions, from campaign plan, a contract review, a financial model, a research synthesis, a content system. The model knows the constraints, the decisions already made, the paths already ruled out. Then something interrupts it. Maybe a session ends or usage cap lands. Someone decides the new model is cheaper or smarter and switches it. Then a colleague picks up the thread. Suddenly a large part of the shared understanding has to be rebuilt from scratch, and the rebuild is slower and worse than the original. That rebuild has a name, once you break it apart, and that’s because these are two different failures often lumped together under the same complaint. Defining Context Rot Context rot is the measured decline in a model’s output quality as the amount of text you feed it grows. It is a property of the model inside a single session, and it shows up well before the context window is anywhere near full. The term comes from a 2025 Context Rot report https://research.trychroma.com/context-rot by the research team at Chroma, who ran 18 frontier models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3, through deliberately simple tasks at increasing input lengths. Every model tested got less reliable as the input grew. Not some of them. All of them. The mechanism is not new. Back in 2023, a Stanford-led team documented what they called the lost in the middle https://arxiv.org/abs/2307.03172 effect, where models use information best when it sits at the very start or the very end of the input, and their accuracy drops when the thing they need is buried in the middle of a long context. The Chroma work extends that finding. Vendors advertise million-token context windows as if more room means more understanding. In real use, stuffing more into the window can make the answer much worse. So the first thing eating your context is the model itself, quietly, as your working session gets longer. That is context rot. It is real, it is measurable, and it is not the whole story. The Double Decay The second thing is context loss, and it is an operations problem rather than a model property. Context rot happens inside one session. Context loss happens when a session ends and a new one starts cold. It could happen when a usage limit forces a stop, when a team switches from one model or tool to another, or when a person hands the work to a colleague. The model retains nothing across those seams unless something outside the chat carries it over. Confusing the two leads to the wrong fix. If your problem is context rot, a shorter, cleaner prompt helps. If your problem is context loss, no amount of prompting inside a dead session brings the knowledge back, because the knowledge was never written down anywhere it could survive. It also helps to be specific about what “context” contains, because the different parts survive a switch very unequally. Files and prior outputs are easy to move. The reasoning behind a decision is the part that dies. | Type of context | What it holds | Survives a switch? | |---|---|---| Factual | Documents, data, prior outputs, links | Usually, if you re-attach it | Decision | Choices already made, options rejected, and the reasons for both | Rarely, this is the first to go | Constraint | Hard rules, brand voice, legal boundaries, risk appetite, style guides | Only if re-injected deliberately | Process | Current state of the work, next intended steps, open questions | Partly, and often out of date | Preference | How the operator likes to work, what has been tried, the model’s calibration to this project | Almost never | Notice the pattern down the right column. The context that is cheapest to store is the context that stays, and the context that made the work good is the one that gets lost. A new session can re-read the brief. It cannot re-read the argument you had three sessions ago about why the obvious approach was wrong. The Subscription Fee for Forgetting The cost is easy to feel and hard to see on a balance sheet, which is why it accumulates. Three bodies of evidence give it a shape. The first is the cost of reorientation. When focused work is interrupted, getting back to full engagement takes time. The most cited figure, roughly 23 minutes to fully refocus, is associated with Gloria Mark’s research at UC Irvine https://ics.uci.edu/~gmark/chi08-mark.pdf , and note that that the precise “23 minutes and 15 seconds” number people repeat comes from her interviews rather than the printed paper. The paper itself shows something arguably worse for quality. It found that interrupted workers finish tasks faster but under measurably higher stress and effort. A model switch is an interruption with extra steps, because you are recovering both your focus and the machine’s. The second is the cost of switching between tools. A 2021 study by Qatalog and Cornell University’s Ellis Idea Lab https://venturebeat.com/business/qatalog-people-waste-59-minutes-every-day-trying-to-find-data-in-apps , surveying a thousand workers , found it takes about 9.5 minutes on average to get back into a productive workflow after toggling to a different application , and that people were losing close to an hour a day just searching for information scattered across their tools , That research predates the current wave of AI tools, so treat it as directional rather than exact , Even so, the direction is not encouraging : every new surface your team adds , is another seam where context leaks. The third is the cost of knowledge that lives in one place and then leaves. Panopto’s workplace knowledge report https://www.panopto.com/company/news/inefficient-knowledge-sharing-costs-large-businesses-47-million-per-year/ found that 42% of institutional knowledge is unique to the individual who holds it, and that workers waste an average of 5.3 hours a week either waiting on information or recreating knowledge that already existed. Those numbers are from 2018 and describe human-to-human knowledge loss, but the shape maps cleanly onto AI work. The custom instructions, the prompt patterns that clicked after multiple iterations, the tacit calibration between a person and a model, all of that is institutional knowledge and it ends up living in a chat history. Moreover, prompts do not travel well. It means when a prompt tuned for one model is moved to another, its performance can degrade by more than 30% https://arxiv.org/abs/2509.03117 , even when it was explicitly built to be portable. And switching is now the norm, not the exception. Andreessen Horowitz found that 37 https://a16z.com/ai-enterprise-2025/ % of enterprises run five or more models https://a16z.com/ai-enterprise-2025/ in production, and a Dataiku and Harris Poll survey of 600 CIOs reported that 81% expect to depend on two or more providers https://www.dataiku.com/blog/ai-switching-problem in 2026, including what the model surfaces to users https://industrycontents.com/ai-mention-vs-recommendation/ . Every one of those switches is a seam. The GPT-5 rollout in August 2025 made the human side of this visible when OpenAI briefly removed access to older models https://www.platformer.news/gpt-5-backlash-openai-lessons/ and people found the workflows they had built around a specific model’s behaviour simply stopped working overnight. None of these figures is a headline you can put on an invoice. Added together across a team that runs AI work every day, they are a standing tax on output. This is the same accounting problem that turns a small per-token inefficiency into a real budget line, which we pulled apart in the piece on how a long conversation costs more than a short one https://industrycontents.com/ai-token-costs-budget-overrun/ . The Fixes You’ve Already Tried The AI guides are full of standard recommendations , and they are not wrong. They are just incomplete, and it helps to see where each one runs out. Start with writing a handoff note. Necessary, and better than nothing. But a note written in a hurry , captures what got done, not why the rejected options were rejected. Reasoning is the expensive part, and it is the part a rushed summary drops first. Then using projects, custom instructions, or the memory feature , Genuinely useful inside one vendor’s ecosystem , The moment you switch tools , that scaffolding does not come with you, and the more you rely on it, the more expensive the eventual switch becomes. Probably the most common fix of them all : summarising the chat. A summary is only as good as its author’s judgment , about what mattered , and it usually flattens the decision trail into a list of outcomes , The context rot problem also comes into play here. When you paste a huge summary into a fresh session when you are back to a long input that the model reads unevenly. Starting a new chat with the full context , This assumes the bottleneck is access, when the real bottleneck is the quality of what you carry over. A wall of pasted history triggers the loss in the middle effect and buries the three sentences that actually needed to survive. The Context Debt Audit You cannot manage what you have not measured, and most teams have never put a number on this. The Context Debt Audit is a lightweight way to do it. Run it for two weeks on any workstream that spans more than one session. Log four numbers each time you resume work after a break, a switch, or a handoff. - Minutes from reopening the work to your first real piece of new progress. This is your reorientation tax, and it is the single most important number. - A tally of choices you had to explain again because the model or the new session did not know them. - Count the outputs that broke a rule you had already established, such as a style guide, a legal boundary, or a format. - Roughly, what fraction of the session went to getting back up to speed versus making new progress. Two weeks of this gives you four trend lines to act on. It also tells you which of the two problems you have. High reorientation time inside long single sessions points at context rot. High restated-decision and constraint-slip counts at the seams point at context loss. The fix is different for each , and that’s the whole reason this audit matters. Structural Patterns for Context Preservation The first thing that helps is to stop keeping context inside conversations and start keeping it in artifacts. Anthropic’s own engineers landed on the same idea for long-running agents. In a 2025 write-up on building effective harnesses https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents , they compared the problem to a software project staffed by engineers working in shifts, where each new shift arrives with no memory of the last, and the fix was to have the work write down its own state so the next session can pick it up. The same logic applies whether the next session is a different model, a different tool, or a different person. - Keep a context object. One short living brief that holds the goal, the current state, and the hard constraints, stored outside any chat. It travels with the work and makes the conversation disposable, which is how it should be. Think of a context object like a sticky note that holds everything important about your project. It tells you what you are trying to do, where you are right now, and the rules you cannot break. You keep this note outside of any chat, in a place you control. Every time you start a new conversation with an AI, you paste it in. That way, you do not have to explain yourself over and over, and the AI gets straight to work. The best part is that because the real plan lives on your note, the chat itself becomes throwaway. You can close it, lose it, or start a fresh one anytime, and nothing really changes. The work keeps moving forward, not because of any one conversation, but because the context travels with you. - Always log decisions separately from chat. A short running record of what was chosen, what was rejected, and why. This is the single highest-value habit, because the rejected paths are what a summary always loses. - Try to maintain a constraint registry. The non-negotiable rules in one place, ready to be pasted back in deliberately at the start of any new session or tool. Do not trust the model to remember them across a seam. - Snapshot state at natural breakpoints. When you finish a phase, write the two or three sentences a stranger would need to continue. Do it at the breakpoint, not when you are already locked out and working from memory. - Run a survival check. After a switch, paste the context object into the new tool and ask it to restate the goal, the constraints, and the last decision. If it cannot, the object is incomplete, and you have found the gap. The reason to favour patterns that live outside any one product is the same reason lock-in is a risk in the first place. If your continuity depends on a vendor’s memory feature, then treating context as a finite resource you manage yourself https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents is what keeps a model switch from becoming a migration project. Beyond the Current Fix It would be dishonest to end on a tidy note, because the underlying problem is not solved. Truth be told, no mainstream tool today gives you continuity that holds across models, across sessions, and across multiple people at professional depth. Token limits, the uneven attention that produces context rot, and the commercial incentive to keep you inside one ecosystem all push against it — especially control over which AI agents your content appears in https://industrycontents.com/brand-safety-blocklist-ai-agents/ . Some context resists being written down at all. The tacit calibration between an experienced operator and a model they have used for months, the feel for how to phrase a request so it lands, is exactly the preference context in the table above, and it is the hardest to externalise. The patterns here reduce the debt. They do not make it zero, and anyone selling a tool that claims to should be read with that in mind. Where to Start This Week Pick one workstream that already spans several sessions and do three things. Start a single context object for it, in a plain document outside your AI tool. Run the Context Debt Audit on it for two weeks and actually write the four numbers down. And the next time a session hits a wall or you switch models, run the survival check before you trust the new session with anything that matters. None of this needs new budget or new software, and the same discipline is what separates a short pause from a lost afternoon when a session hits its limit mid-project https://industrycontents.com/claude-usage-limit-mid-project/ . Context is the asset your team is building every time it works with a model. Right now, for most teams, it is an asset with no owner and no shelf. Giving it one is the cheapest quality improvement available. FAQ Context rot is the measured decline in a language model’s output quality as the amount of text you feed it grows. It appears even on simple tasks and well before the context window is full. A 2025 Chroma study tested 18 frontier models and found every one became less reliable as its input got longer. No. Context rot happens inside a single session as input grows. Losing context when you switch models, tools, or sessions is a separate operations problem: the model retains nothing across those seams unless you carry it over deliberately. The fixes differ, which is why the distinction is worth keeping. Prompts are tuned to a specific model’s behaviour and do not transfer cleanly. Studies of prompt transfer have recorded meaningful accuracy drops, in some cases around a third, when a prompt is moved to a different model, though the size of the drop varies widely by model and task. Keeping your rules in a separate registry, ready to re-inject and re-test, turns that surprise into a quick check. Run the Context Debt Audit for two weeks on any workstream that spans multiple sessions. Each time you resume after a break, switch, or handoff, log four numbers: time to productive continuation, restated decisions, constraint slips, and the share of the session spent reorienting. The trend lines tell you whether your problem is in-session rot or cross-session loss. It helps inside that one tool, but it does not travel. If your continuity depends on a single vendor’s memory feature, switching tools later becomes more expensive, and the decision reasoning that makes work good is often still lost. Storing context in artifacts you control, outside any chat, is what holds up across models and tools.