Context Engineering vs Prompt Engineering: The Winner May Surprise AI Engineers

Andrej Karpathy's June 25, 2025 post endorsing 'context engineering' over 'prompt engineering' has reshaped AI engineering, with Gartner reportedly declaring context engineering the new standard by 2026. A July 2025 Chroma study of 18 frontier models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3, found that every model suffered 'context rot,' becoming less reliable as input length grew, undermining the promise of million-token windows. The shift is driven by the iterative nature of agent systems, where context curation at each turn is critical, as formalized by LangChain's Lance Martin into a four-part taxonomy: write, select, compress, and isolate.

On June 25, 2025, Andrej Karpathy posted eleven words on X that quietly ended one era of AI engineering and opened another. Quoting a post from Shopify CEO Tobi Lütke from a week earlier, Karpathy said he preferred “context engineering” to “prompt engineering,” calling it the delicate art of filling the context window with exactly the right information for the next step. Within days, the term had colonized engineering blogs, conference talks, and LinkedIn bios. Within a year, Gartner was reportedly telling enterprise AI leaders that context engineering was in and prompt engineering was out. That framing has calcified into a consensus most AI engineers now repeat without examining: prompt engineering was the training-wheels discipline of 2023, context engineering is the real skill of 2025 and 2026, and anyone still optimizing individual prompts is behind the curve. It’s a tidy story. It’s also missing the actual plot twist that’s been building in production agent systems since late 2025 — one that neither the prompt engineering loyalists nor the context engineering evangelists saw coming. Prompt engineering, at its narrowest, is the discipline of crafting the wording, structure, and examples inside a single instruction to a model — zero-shot framing, few-shot examples, chain-of-thought scaffolding, output-format constraints. Between 2022 and 2024, it carried outsized weight because most LLM use cases were single-turn: classification, summarization, one-shot generation. The prompt was effectively the entire input, so optimizing the prompt was optimizing the system. Simon Willison, the developer and AI commentator who has tracked this shift as closely as anyone, put his finger on why the term soured: prompt engineering had been diluted into something that sounded like typing hacks into a chatbot, even though the underlying discipline of writing precise instructions for a model never stopped mattering. The word got cheap. The skill didn’t. Anthropic’s Applied AI team gave the discipline its clearest definition in a September 29, 2025 engineering post: prompt engineering is about writing and organizing instructions, while context engineering is the broader set of strategies for curating and maintaining the full set of tokens an LLM sees at inference time — system instructions, tool definitions, retrieved documents, conversation history, memory, everything. Where prompt engineering is a discrete task you complete once, context engineering is iterative : you re-curate what enters the window at every single turn of an agent loop. LangChain’s Lance Martin formalized the practitioner side of this into a four-part taxonomy that has become close to a working standard: write persist information outside the context window, in scratchpads or memory files , select pull the right subset back in via retrieval or tool calls , compress summarize or trim so only high-signal tokens remain , and isolate keep unrelated context in separate windows, typically via sub-agents, so it doesn’t cross-contaminate . Nearly every production agent architecture shipping in 2026 — LangGraph pipelines, Claude Agent SDK deployments, custom harnesses — implements some variant of these four moves. Context engineering’s ascendance isn’t just branding; it has real empirical backing, and this is the part every AI engineer should actually internalize regardless of which side of the naming debate they’re on. In July 2025, Chroma researchers Kelly Hong, Anton Troynikov, and Jeff Huber published a technical report evaluating eighteen frontier models — including the GPT-4.1, Claude 4, Gemini 2.5, and Qwen3 families — on how performance changes as input length grows. They coined the term “context rot” for what they found: contrary to the marketing promise of million-token context windows, every single model they tested got measurably less reliable as more tokens were added, well before hitting any stated limit. The degradation wasn’t linear or predictable across models — some held steady past 32K tokens and then fell off a cliff, others degraded gradually — and, in a genuinely counterintuitive result, models performed worse on logically coherent documents than on shuffled, unstructured ones, suggesting that narrative coherence itself creates more convincing distractors for the attention mechanism to get lost in. Anthropic’s own explanation ties this to architecture, not just training data: transformer attention creates n² pairwise relationships between tokens, so as context length grows, a model’s ability to track every relevant relationship gets stretched thinner. Models also see far more short sequences than long ones during training, so they have proportionally less experience with genuinely long-range dependencies. The upshot, in Anthropic’s phrasing, is that LLMs have an “attention budget” much like human working memory has a limited capacity — every additional token spends down that budget, whether or not the token is useful. This is the load-bearing argument for context engineering: if models degrade with volume regardless of the words used to invoke them, then the highest-leverage move isn’t perfecting your instruction, it’s minimizing and curating everything sitting alongside it . A 2026 industry survey on enterprise AI adoption found a large majority of IT and data leaders now agree prompt quality alone can’t carry production AI reliability at scale — a striking reversal from the prompt-obsessed framing of just two years earlier. By early 2026, “context engineering replaced prompt engineering” had become the kind of claim nobody bothered to fact-check anymore. It shows up in consulting decks, conference keynotes, and no small number of Medium posts with a CPU-and-RAM diagram borrowed from Karpathy’s own analogy: the model is the processor, the context window is RAM, and context engineering is the discipline of managing what gets loaded into that limited working memory. The trouble with treating this as the final word is that it stopped at exactly the point where the actual production evidence was starting to get interesting. Here’s the uncomfortable fact for anyone who has spent 2025 and 2026 building elaborate retrieval pipelines and context-curation systems: some of the most dramatic reliability gains reported in production agents over the past year came from neither better prompts nor better-curated context. They came from re-engineering the environment the agent operates inside — a layer now being called harness engineering , and it is quietly eating both of its predecessor disciplines. Two incidents make the case better than any framework diagram. The first comes from Cognition, the company behind the autonomous coding agent Devin. While rebuilding Devin’s architecture around Claude Sonnet 4.5, Cognition’s team documented a failure mode they called “context anxiety” : as the context window filled up, the model would sense the limit approaching and start wrapping up tasks prematurely — cutting corners not because the work was finished, but because it had internalized a sense of running out of room. No amount of prompt rewording or context curation fixed this, because the problem wasn’t the content of the context — it was the model’s awareness of its own constraint. Cognition’s fix was pure environment design: they enabled the 1-million-token context beta but silently capped actual usage at 200,000 tokens , giving the model the psychological experience of having ample runway even though the effective working set hadn’t changed. The anxiety disappeared. Nothing about the prompt or the retrieved context changed at all. The second comes from LangChain’s own engineering team. In March 2026, they reported moving their coding agent from 30th place to 5th place on the Terminal-Bench 2.0 leaderboard without changing the underlying model at all. The entire gain came from optimizing the harness — the scaffolding of tool design, execution sandboxing, retry logic, and verification loops surrounding the model — while the intelligence doing the reasoning stayed identical. Neither of these are prompt engineering wins. Neither are, strictly, context engineering wins in the write/select/compress/isolate sense — no document got better-curated, no retrieval pipeline got smarter. What changed was the operating environment : how the agent perceives its own constraints, how tool failures get handled, how work gets verified before it’s accepted, how sub-tasks get scoped and sandboxed. Cognition’s Walden Yan, who was writing about the discipline before Karpathy’s tweet made “context engineering” a household term, has been one of the loudest voices arguing that as of 2026, the harness — not the prompt, not even the context payload — is what separates agents that work in demos from agents that survive production. This is where the tidy prompt-versus-context narrative breaks down completely, and it’s worth being precise about why, because getting this wrong costs teams real engineering time. Prompt engineering did not get replaced. It got absorbed — and then, one layer up, absorbed again. Anthropic’s own framing makes this explicit: context engineering is described as the natural progression of prompt engineering, not its replacement, and system prompts remain one of the components context engineering has to curate. Every context-engineered system still has to write a system prompt, and Anthropic’s guidance on getting that prompt right — finding the “right altitude” between brittle, over-specified logic and vague, under-specified guidance — is prompt engineering in every meaningful sense of the term, just practiced with more architectural awareness of what surrounds it. The nested structure repeats one level further out. A well-designed harness still needs well-designed tool descriptions, which are still prompts. A context-curation pipeline still needs a compaction prompt tuned for recall and precision, which Anthropic explicitly recommends “carefully tuning” on real agent traces — that is prompt engineering, applied to the compression step of context engineering, running inside a harness. The disciplines don’t compete for the same job. They stack. If you strip away the branding wars, what’s actually happened between 2023 and 2026 is a steady widening of the unit of control AI engineers are responsible for: The “winner” implied in the debate’s framing was never going to be prompt engineering or context engineering alone, because the debate itself was asking the wrong kind of question — a single-layer question about a problem that had already become a three-layer stack by the time the argument caught on. The actual surprise, for engineers who spent 2025 racing to rebuild their retrieval pipelines because Gartner said prompting was “out,” is that the discipline delivering the sharpest production gains in 2026 sits a level above the one everyone just finished migrating to. None of this is an argument for abandoning context engineering — the Chroma research on context rot is real, reproducible, and holds across every model family tested, so treating context as a finite, carefully curated resource remains non-negotiable for anyone building agents that run longer than a single turn. But it is an argument against stopping there. Three things worth auditing this quarter: whether your team still treats prompt quality as an afterthought once “the context pipeline is done” — it isn’t, because every compaction step, tool description, and sub-agent instruction is still a prompt that can be tuned; whether your context-curation investment has a matching investment in evaluation and verification loops , since a perfectly curated context window feeding an ungoverned agent loop is still an ungoverned agent loop; and whether anyone on the team is actually looking at how the agent behaves as it approaches its own operational limits, because that failure mode won’t show up in a context-quality audit at all. The prompt-versus-context debate produced a genuinely useful vocabulary. It also produced a genuinely misleading scoreboard. The engineers who benefit most in 2026 won’t be the ones who picked a side — they’ll be the ones who noticed the argument was really describing three layers of the same problem, and quietly started building for all three. Context Engineering vs Prompt Engineering: The Winner May Surprise AI Engineers https://pub.towardsai.net/context-engineering-vs-prompt-engineering-the-winner-may-surprise-ai-engineers-652d7e991448 was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.