For the last few years, building with large language models often started with one question:
“What prompt should I give the model?”
Developers experimented with system prompts, role instructions, few-shot examples, XML tags, Markdown formatting, and increasingly elaborate instructions.
And it worked.
A better prompt could turn an unreliable output into a surprisingly useful one.
But modern AI applications are becoming more complex.
We are no longer only asking an LLM to summarize a paragraph or generate an email. We are building AI agents that search databases, call APIs, remember previous conversations, read documents, use tools, execute code, and work across multiple steps.
In these systems, writing a good prompt is only one part of the problem.
The bigger question becomes:
What information should the model have access to at this exact moment?
That is the problem context engineering tries to solve.
Anthropic describes context engineering as a natural progression from prompt engineering: instead of focusing only on the instructions written inside a prompt, developers manage the entire set of information available to the model during inference.
And that shift changes how we think about building AI applications.
Prompt engineering is the practice of designing instructions that help an LLM produce the behavior or output we want.
For example:
You are a senior Python developer.
Review the following code for:
- security issues
- performance problems
- readability
Return your answer as:
1. Issue
2. Why it matters
3. Suggested fix
Code:
{code}
There is nothing wrong with this.
In fact, good prompts remain extremely important.
The developer has clearly defined:
For relatively isolated tasks, that might be enough.
But imagine turning this into an AI coding assistant.
Now the model may also need to know:
You could technically throw everything into one gigantic prompt.
But that creates another problem.
More context does not automatically mean better context.
Context engineering is the process of deciding what information an AI model receives, when it receives it, and how that information is structured.
LangChain describes the idea as providing the right information and tools in the right format so that an LLM can successfully complete its task.
Think of the difference this way:
Prompt engineering asks:
How should I phrase the instruction?
Context engineering asks:
What does the model need to know before it can correctly follow that instruction?
That context might include much more than the user's prompt.
For example:
Context
│
├── System instructions
├── User message
├── Conversation history
├── Retrieved documents
├── Long-term memory
├── Few-shot examples
├── Available tools
├── Tool descriptions
├── Tool results
├── Application state
├── User preferences
└── Output requirements
The prompt is still there.
It simply becomes one component of a much larger context architecture.
Imagine we are building an AI support assistant for an e-commerce store.
A prompt-engineering approach might look like this:
prompt = f"""
You are a helpful customer support agent.
Answer the customer's question politely.
Customer:
{message}
"""
response = llm.generate(prompt)
Suppose the customer asks:
Where is my order?
The prompt is perfectly reasonable.
But the model cannot give a useful answer.
Why?
Because it has no idea:
No amount of rewriting:
Be extremely helpful.
or:
Think carefully before answering.
can magically give the model information it does not have.
Instead, the application needs to assemble relevant context.
Conceptually, the system might do something like:
customer = get_customer(user_id)
order = get_latest_order(customer.id)
shipment = get_shipment_status(order.id)
context = {
"customer": customer,
"order": order,
"shipment": shipment
}
response = llm.generate(
instructions=SUPPORT_INSTRUCTIONS,
user_message=message,
context=context
)
Now the model might receive:
Customer:
Alex
Order:
#81452
Status:
Shipped
Courier:
FedEx
Estimated delivery:
August 12
Latest tracking event:
Package arrived at regional facility
Suddenly, answering “Where is my order?” becomes straightforward.
The important improvement was not a cleverer sentence inside the prompt.
It was better context.
The change is closely tied to how AI applications themselves are evolving.
Early LLM applications were often simple:
Input → LLM → Output
Modern agentic applications can look more like:
User
↓
Agent
↓
Search documentation
↓
Read database
↓
Call API
↓
Evaluate result
↓
Call another tool
↓
Update state
↓
Generate response
An agent may repeatedly call the model and use tools until it completes the task. LangChain's current agent documentation describes this basic loop as alternating between model calls and tool execution.
Every step generates more information.
Tool responses accumulate.
Conversation history grows.
Documents get retrieved.
Intermediate reasoning creates new state.
Eventually, the problem is no longer merely:
“How do I instruct the model?”
It becomes:
“Which pieces of all this information should be present for the next model call?”
That is a context-engineering problem.
Modern models can process very large context windows, but developers should not treat them as databases where everything should simply be dumped.
Anthropic notes that model performance can degrade as context grows and describes context as a finite resource with diminishing returns. Relevant information therefore needs to be carefully selected rather than indiscriminately accumulated.
This creates an important rule for AI developers:
The goal is not maximum context. The goal is useful context.
Imagine asking a developer to fix one function in a large repository.
Giving them the relevant function, its tests, related interfaces, and the current error would probably help.
Printing the entire company codebase, every Slack message ever sent, six years of Git history, and all internal documentation onto their desk probably would not.
LLMs face a similar information-management problem.
Extra information can create:
Recent OpenAI engineering guidance similarly discusses avoiding context bloat in agent systems because unnecessary tools, history, and integrations can increase cost and distract the model.
Context engineering therefore involves both adding information and removing information.
You do not necessarily need a new job title called Context Engineer.
Context engineering is better understood as a skill developers building AI systems increasingly need.
Here are some of the major things you may control.
These are your traditional prompts:
You are a financial document analyzer.
Prompt engineering still matters here.
Instead of putting an entire knowledge base into the prompt, your application can retrieve relevant information when needed.
For example:
User question
↓
Search knowledge base
↓
Retrieve relevant documents
↓
Add documents to context
↓
LLM generates answer
This is one reason retrieval-augmented generation, or RAG, became such an important LLM architecture.
A chatbot might have hundreds of previous messages.
The model may not need all of them.
Your application could keep:
Last 10 messages
+
Summary of older conversation
+
Important saved facts
instead of repeatedly passing the entire conversation.
For agents, tools themselves are context.
The model needs to understand what capabilities are available.
For example:
tools = [
search_web,
query_database,
send_email,
create_calendar_event
]
The names, descriptions, parameters, and results of those tools influence what the model decides to do next.
LangChain therefore treats tool availability and tool context as part of the broader context-engineering problem.
Some information should survive beyond a single conversation.
An AI assistant might remember:
Preferred programming language: TypeScript
Project framework: Next.js
Database: PostgreSQL
Deployment: AWS
Instead of keeping every previous conversation in the context window, the application can store useful information externally and retrieve it when relevant.
Tool outputs can become surprisingly large.
Imagine an agent runs:
npm test
and receives 15,000 lines of output.
Does the next model call really need all 15,000 lines?
Probably not.
A better system may extract:
Tests failed: 3
Failures:
- auth.test.ts: token expiration mismatch
- cart.test.ts: incorrect subtotal
- checkout.test.ts: missing address validation
That is context engineering.
The model receives the signal, not all the noise.
A useful mental model presented by LangChain groups common context-engineering techniques into four categories: write, select, compress, and isolate.
Store information outside the immediate context so it can be used later.
Examples include:
Retrieve only information relevant to the current task.
For example:
documents = search(
query=user_question,
limit=5
)
instead of 5,000 documents.
Reduce large amounts of information while preserving what matters.
For example:
120-message conversation
↓
Structured summary
↓
Current context
Anthropic and OpenAI both describe compaction techniques for long-running agents where accumulated history is reduced into smaller representations that preserve important state.
Keep unrelated work in separate contexts.
Instead of making one agent carry everything, specialized agents might handle different tasks:
Main Agent
│
├── Research Agent
├── Coding Agent
└── Testing Agent
Each agent gets the context needed for its specific job and can return a concise result to the coordinator.
Anthropic discusses this approach for complex agent workflows as a way of preventing detailed subtask information from consuming the primary agent's context.
The easiest way to understand the transition is to compare them directly.
| Prompt Engineering | Context Engineering |
|---|---|
| Optimizes instructions | Optimizes the model's information environment |
| Focuses mainly on prompts | Manages prompts, memory, tools, retrieval and state |
| Often static | Usually dynamic |
| Common in single LLM calls | Critical in multi-step agents |
| Asks “How should I say this?” | Asks “What should the model know?” |
| Still useful | Includes prompt engineering as one component |
So saying context engineering is replacing prompt engineering requires a little nuance.
Prompt engineering is not disappearing.
Its role is becoming smaller relative to the rest of the system.
Anthropic explicitly describes context engineering as the natural progression of prompt engineering rather than its complete replacement.
This may be the most important takeaway.
Building reliable AI applications increasingly looks less like discovering magical prompt phrases and more like traditional software engineering.
Developers need to think about:
Data
↓
Retrieval
↓
State
↓
Memory
↓
Permissions
↓
Tools
↓
Context
↓
Model
↓
Validation
The LLM sits inside a system.
Its output depends heavily on what that system makes visible to it.
Consider two identical models.
Receives:
Help the user debug their application.
Receives:
Relevant source files
Current stack trace
Dependency versions
Project architecture
Recent code changes
Available terminal tools
Team coding standards
User's actual question
Even if both models are equally intelligent, System B has a massive practical advantage.
Not because its prompt contains better adjectives.
Because its information environment is better engineered.
Definitely not.
A poorly written instruction can still produce poor results.
Developers still need to understand:
But those skills now belong inside a bigger discipline.
The progression looks something like this:
Prompt Engineering
↓
Prompt + Retrieval
↓
Prompt + Retrieval + Memory
↓
Prompt + Tools + State + Memory
↓
Context Engineering
As AI applications move from single-turn generators toward agents capable of working across tools and longer-running tasks, managing that context becomes increasingly central to system reliability.
When your AI system produces a bad answer, resist immediately changing the prompt.
Instead, ask:
Did the model receive the information
required to make the correct decision?
Then investigate:
Sometimes the solution will still be a better prompt.
But increasingly, the solution will be better context architecture.
Prompt engineering taught developers how to communicate with language models.
Context engineering asks us to go one level deeper and design the environment in which those models operate.
For simple LLM applications, a carefully designed prompt may still be most of what you need.
For modern AI agents, however, the model may depend on retrieved documents, tools, memory, application state, conversation history, intermediate results, and runtime information.
Someone has to decide what gets included.
Someone has to decide what gets removed.
Someone has to decide what the model should know at each step.
That is context engineering.
And as AI development moves from:
Prompt → Response
toward:
Context → Model → Tool → State → Context → Model → Action
the developers who understand how to engineer that context will have a much better mental model for building reliable AI systems.
The future of AI development isn't about finding the perfect prompt.
It's about giving the model the right information, at the right moment, in the right form.