Context Engineering: Why Your AI Agent Needs a Database, Not a Prompt

Published: August 22, 2026 | Focus Keyword: context engineering for AI agents | Est. read time: 14 minutes

You've built the agent. It passes every eval. Then you deploy it.

On day one, it's brilliant. By week three, it's recommending a customer return a product they've already returned twice before, referencing a policy that changed six weeks ago, and confidently calling an API endpoint that was deprecated in the last sprint. You've tuned the prompt a hundred times. You've tried longer system prompts, few-shot examples, chain-of-thought. The agent is still stuck at 24% accuracy on long-horizon tasks.

Here's the uncomfortable truth: the model isn't the problem. The context is.

This is the inflection point the ML engineering community hit in mid-2026. When OpenViking — VolcEngine's open-source context database for AI agents — became the #1 trending Python repository on GitHub, it wasn't because engineers were excited about another RAG wrapper. It was because they recognised something more profound: the problem of agent memory had outgrown the vocabulary of prompting. It had become a database problem.

Context engineering for AI agents is the emerging discipline of designing and managing the information environment in which your agent operates — not as a static prompt, but as a living, structured, tiered data system. Done right, it transforms that 24% agent into one running at 82%.

This post is the technical deep-dive you need to understand why, and how to build it.

Before we talk about the solution, let's precisely name the problem. Every AI agent draws on some combination of six context primitives:

The text passed directly in the prompt. Fast, zero-latency, but brutally limited. A 1M token window sounds like infinite space until you're running a multi-day coding agent across a 500K-line codebase. And crucially, not all tokens in a long context are attended to equally — the "lost in the middle" problem means your critical instructions buried at position 300K may as well not exist.

The standard fix — embed your knowledge base, retrieve the top-k chunks at query time. RAG is essential, but it fails in two ways: precision collapses on multi-hop queries (asking about a relationship between two entities that each live in separate chunks), and it has no memory of what it already retrieved. Every turn is stateless.

Real-time grounding via search or APIs. Excellent for current events, terrible for internal knowledge. And as the August 2026 UK AISI incident report showed, agents with live web access in improperly sandboxed environments can cause real damage.

Structured, typed callable functions. The Model Context Protocol (MCP) has standardised this. But skills are stateless by design — they do one thing, return a result, and forget. They don't accumulate knowledge across invocations.

The chat history buffer. This is the scratchpad that every agent has, but it's ephemeral — it dies with the session. It also grows unboundedly until it hits your context limit, at which point you truncate it and lose the beginning of your reasoning chain.

The piece almost everyone gets wrong. Most teams implement this as "save embeddings of conversation turns to a vector database." This is better than nothing, but it's a poor approximation of what agents actually need.

class NaiveAgentMemory:
    def __init__(self, vector_db):
        self.db = vector_db

    def save(self, turn: str):
        embedding = embed(turn)
        self.db.upsert(embedding, metadata={"text": turn})

    def recall(self, query: str, top_k: int = 5) -> list[str]:
        results = self.db.query(embed(query), top_k=top_k)
        return [r.metadata["text"] for r in results]

The problem is structural: you're using a search engine to solve a database problem. A search index answers "what text is similar to this query?" A database answers "what is the state of this entity, what changed, when, and why?"

The four storage forms that together constitute a complete agent context database. Each serves a distinct access pattern — no single form is sufficient alone.

The OpenViking framework, whose VikingMem paper was accepted to VLDB 2026 (the top database systems conference), defines context engineering for AI agents around four complementary organization forms. Think of them as the four tables in your agent's relational schema:

What it's good at: fuzzy recall, concept-level retrieval, semantic search across unstructured text.

What it's bad at: precise lookups, relational joins, structured queries.

When to use it: retrieving relevant past episodes, similar code patterns, analogous situations.

What it's good at: navigating large knowledge bases with known structure, progressive disclosure, lazy .

What it's bad at: fuzzy search, ad-hoc queries.

When to use it: project documentation, codebase knowledge, anything with a natural tree structure.

OpenViking's viking://

protocol is the most elegant implementation of this pattern — it gives your agent a virtual filesystem address space for all its knowledge, with path-based access that mirrors how humans and IDE tools naturally navigate information.

viking://project/architecture/decisions/adr-042-database-choice.md  # L2: Full ADR
viking://project/architecture/decisions/                             # L1: ADR index
viking://project/architecture/                                       # L0: "project uses PostgreSQL, event sourcing"

What it's good at: precise lookups, aggregations, current state of structured entities.

What it's bad at: unstructured text, semantic search.

When to use it: user profiles, task state, tool call history, API response caches.

-- Agent context as structured state
-- This is what you actually want for entity tracking

CREATE TABLE agent_context_entities (
    entity_id     TEXT PRIMARY KEY,
    entity_type   TEXT NOT NULL,  -- 'user', 'task', 'codebase', 'decision'
    state         JSONB,
    last_updated  TIMESTAMPTZ,
    session_count INT DEFAULT 0,
    confidence    FLOAT  -- agent's confidence in this knowledge
);

CREATE TABLE agent_context_relations (
    from_entity   TEXT REFERENCES agent_context_entities(entity_id),
    relation_type TEXT,
    to_entity     TEXT REFERENCES agent_context_entities(entity_id),
    evidence      TEXT,
    strength      FLOAT
);

What it's good at: multi-hop reasoning, relationship traversal, inferring implicit connections.

What it's bad at: fuzzy lookup, scale (can get expensive for large graphs).

When to use it: reasoning about how concepts, people, decisions, and code artifacts relate to each other.

The combination of all four forms is what transforms a "memory-augmented chatbot" into an agent that genuinely knows things — with structure, provenance, and the ability to update its knowledge as the world changes.

L0 gives the agent orientation (100 tokens). L1 gives structure (2K tokens). L2 provides full detail only when needed — dramatically reducing token consumption and latency.

Understanding what to store is only half the battle. The other half is understanding how much of it to put in the context window at any given moment.

The naive approach: stuff everything into the prompt. Result: slow, expensive, attention-diluted.

The smarter approach: tier your context .

OpenViking's three-tier system is the most rigorous implementation of this pattern:

A compressed, always-present header for each knowledge unit. Think of it as the card in a card catalogue — just enough to know whether this document is relevant without the document itself.

L0 example for a microservice's context entry:
"payment-service: Stripe-based payment processing. Owns /payments/* endpoints. 
Last updated 2026-08-15. 3 known issues. 2 pending breaking changes."

The agent loads ALL L0 summaries for a project at start — total cost: perhaps 5K tokens for a 100-module codebase.

The table of contents plus key facts — loaded when the L0 signals relevance. For a service, this might include its API contract, key dependencies, recent change history, and known issues.

The agent loads L1 only for services that are likely relevant to the current task — cutting irrelevant entirely.

The complete knowledge artifact: full source code, full documentation, full conversation history. Loaded only when the agent needs to reason about specifics.

class TieredContextDB:
    def __init__(self, viking_client):
        self.db = viking_client

    async def load_context_for_task(self, task: str, budget_tokens: int = 8000):
        """Smart tiered  — load only what's needed."""

        l0_summaries = await self.db.load_tier(level=0, scope="all")

        relevant = self.rank_by_relevance(l0_summaries, task, top_k=10)

        l1_details = []
        remaining_budget = budget_tokens - sum(s.token_count for s in l0_summaries)

        for candidate in relevant[:5]:
            if remaining_budget < 2000:
                break
            l1 = await self.db.load_tier(level=1, entity_id=candidate.id)
            l1_details.append(l1)
            remaining_budget -= l1.token_count

        context = ContextBundle(
            always_present=l0_summaries,
            structured_detail=l1_details,
            lazy_=lambda entity_id: self.db.load_tier(level=2, entity_id=entity_id)
        )
        return context

    async def evolve(self, task: str, result: str, agent_trace: list):
        """Self-evolution: update the DB based on what the agent learned."""
        new_knowledge = await self.extract_knowledge(agent_trace)
        await self.db.merge(new_knowledge)  # Viking's conflict-resolution merge
        await self.db.regenerate_summaries(affected_entities=new_knowledge.entities)

The tiering principle maps directly to how experienced engineers actually work: you scan filenames first, read READMEs second, and read source code only when necessary. The difference is your agent now does this systematically, cheaply, and automatically.

The performance gap between naive retrieval and structured context engineering is not incremental — it is categorical. These numbers are from published evaluations on production-grade benchmarks.

Let's be precise about what the numbers actually measure and mean.

LoCoMo is a benchmark specifically designed to test agents on long-running conversational scenarios — the kind where a customer support agent needs to remember a user's history across dozens of sessions, or a coding agent needs to track decisions made three weeks ago.

The 3.39× accuracy improvement is striking. The 91% token reduction is arguably more important for production systems — it's the difference between a context-enriched agent that costs $0.003/query and one that costs $0.033/query. At scale, that's an order of magnitude difference in operational cost.

HotpotQA tests the ability to answer questions that require chaining multiple facts — the bread-and-butter of any non-trivial agent task.

The 326× latency improvement (75s → 0.23s) is not a typo. The structural tiering means OpenViking can answer multi-hop questions by navigating its filesystem-shaped knowledge index rather than running expensive graph traversals or sequential LLM calls. The indexing cost savings (62.7M → 8.67M tokens, an 86% reduction) also dramatically cut the cost of onboarding new knowledge.

tau2-bench tests agents on real-world task completion scenarios in retail and airline customer service — domains with high entity complexity, policy lookups, and state management requirements.

A +11.87 percentage point improvement in a production task completion benchmark is the kind of result that changes quarterly metrics for AI product teams. These are not toy improvements.

Enough theory. Let's build something. The following walkthrough takes you from zero to a context-engineered agent in under 30 minutes.

pip install openviking

viking init my-agent-context
cd my-agent-context

import asyncio
from openviking import Viking, Ingester

async def ingest_codebase():
    viking = Viking(db_path=".viking")
    ingester = Ingester(viking)

    await ingester.ingest_repository(
        path="./src",
        entity_type="codebase",
        chunk_strategy="by_module",      # or "by_file", "by_function"
        generate_summaries=True,         # LLM-generated L0 and L1 summaries
        extract_relations=True,          # Build the knowledge graph
    )

    await ingester.ingest_docs(
        path="./docs",
        entity_type="documentation",
    )

    await ingester.ingest_files(
        pattern="./decisions/adr-*.md",
        entity_type="architecture_decision",
    )

    print(f"Ingested {len(await viking.list_entities())} entities")
    print(f"Built {len(await viking.list_relations())} relations")

asyncio.run(ingest_codebase())
python
import asyncio
from openviking import Viking
from openai import AsyncOpenAI  # works identically with anthropic.AsyncAnthropic

class ContextEngineeredAgent:
    def __init__(self):
        self.viking = Viking(db_path=".viking")
        self.llm = AsyncOpenAI()
        self.session_id = None

    async def start_session(self, session_id: str):
        """Begin a new agent session — loads L0 context automatically."""
        self.session_id = session_id

        self.base_context = await self.viking.session_start(
            session_id=session_id,
            load_tier=0,         # Always-present L0 summaries
            scope="all",         # Across all knowledge entities
        )
        return self.base_context

    async def run(self, user_message: str) -> str:
        """Process a message with full context engineering."""

        enriched_context = await self.viking.get_context_for_query(
            query=user_message,
            session_id=self.session_id,
            l1_top_k=5,           # Load L1 for top 5 relevant entities
            token_budget=12000,   # Hard cap on context tokens
            include_relations=True,  # Add graph edges for multi-hop reasoning
        )

        system_prompt = f"""You are a helpful engineering assistant.

## Project Context (Auto-loaded by Viking Context DB)

### Always-Present Knowledge (L0 — All Entities)
{enriched_context.l0_overview}

### Relevant Detail (L1 — Top Matches for This Query)
{enriched_context.l1_details}

### Active Relations (Knowledge Graph Edges)
{enriched_context.relations}

### Session Memory (What We've Established This Session)
{enriched_context.session_memory}

If you need deeper detail on any entity, call the `load_context` tool with the entity ID.
"""

        response = await self.llm.chat.completions.create(
            model="gpt-5.6-terra",
            messages=[
                {"role": "system", "content": system_prompt},
                {"role": "user", "content": user_message},
            ],
            tools=[{
                "type": "function",
                "function": {
                    "name": "load_context",
                    "description": "Load full (L2) detail for a specific knowledge entity",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "entity_id": {"type": "string", "description": "The entity ID from L0/L1 summaries"}
                        },
                        "required": ["entity_id"]
                    }
                }
            }]
        )

        if response.choices[0].finish_reason == "tool_calls":
            tool_call = response.choices[0].message.tool_calls[0]
            entity_id = eval(tool_call.function.arguments)["entity_id"]

            l2_content = await self.viking.load_tier(level=2, entity_id=entity_id)


        agent_response = response.choices[0].message.content

        await self.viking.evolve_from_turn(
            session_id=self.session_id,
            user_message=user_message,
            agent_response=agent_response,
            auto_merge=True,      # Automatically merge new facts into the DB
            confidence_threshold=0.85,  # Only merge high-confidence extractions
        )

        return agent_response

async def main():
    agent = ContextEngineeredAgent()
    await agent.start_session("engineering-session-001")

    response = await agent.run("Why did we choose PostgreSQL over MongoDB for the payments service?")
    print(response)

asyncio.run(main())

After running several sessions, inspect how the context DB has evolved:

viking status


viking history --entity payment-service

viking provenance "payment-service uses Stripe"

Production AI deployments in 2026 face a compliance requirement that most context engineering discussions skip entirely: auditability. If your agent makes a decision — recommends a refund, blocks an account, generates a contract clause — you need to be able to reconstruct exactly what context it had when it made that decision.

Semantica — another trending GitHub project this week — addresses this with a graph-native governance layer built on:

from semantica import SemanticaGraph, ProvenanceTrace, SHACLValidator

class AuditableContextDB:
    def __init__(self, viking_client, semantica_graph):
        self.viking = viking_client
        self.graph = semantica_graph
        self.validator = SHACLValidator(schema_path="schemas/agent-context.shacl.ttl")

    async def merge_with_provenance(self, new_knowledge: dict, session_id: str):
        """Merge new knowledge with full PROV-O provenance tracking."""

        validation_result = self.validator.validate(new_knowledge)
        if not validation_result.conforms:
            raise ContextValidationError(
                f"Knowledge rejected: {validation_result.violations}"
            )

        provenance = ProvenanceTrace(
            activity_id=f"merge-{session_id}-{timestamp()}",
            agent_id="context-engineering-agent-v2",
            used=[session_id],         # Which session generated this
            generated_at=datetime.utcnow(),
            confidence=new_knowledge.get("confidence", 0.0),
        )

        await self.graph.merge(
            triples=new_knowledge["triples"],
            provenance=provenance,
        )

        await self.viking.sync_from_graph(self.graph, affected_entities=new_knowledge["entities"])

    async def explain_decision(self, decision_id: str) -> str:
        """Full audit trail for a specific agent decision — SPARQL query."""

        query = f"""
        PREFIX prov: <http://www.w3.org/ns/prov#>
        PREFIX agent: <https://your-org.com/agent-ontology#>

        SELECT ?fact ?source ?session ?timestamp ?confidence
        WHERE {{
            agent:decision-{decision_id} agent:usedFact ?fact .
            ?fact prov:wasAttributedTo ?source .
            ?fact agent:extractedInSession ?session .
            ?fact prov:generatedAtTime ?timestamp .
            ?fact agent:confidence ?confidence .
        }}
        ORDER BY DESC(?timestamp)
        """

        results = await self.graph.sparql(query)
        return self.format_audit_trail(results)

The value proposition for enterprise teams is clear: when the compliance team asks "why did the agent recommend X?", you can produce a timestamped chain of evidence rather than a shrug.

Here's the conceptual shift that takes this from a useful library to a career-defining paradigm:

The old model: Hire ML engineers to fine-tune models, prompt engineers to craft system prompts, and DevOps to deploy them. The model is the product.

The new model: The model is a commodity. The context infrastructure is the product. The engineers who build, maintain, and evolve context databases — who define tiering strategies, self-evolution policies, provenance schemas, and conflict resolution logic — are the ones generating leverage.

This maps directly to the emergence of SRE as a discipline: when compute became cheap and reliable, the engineers who operationalised that reliability at scale became the most valuable people in the room. Context engineering is that moment for AI agents.

What does a "Context Engineer" actually do?

Context Engineer Responsibilities (2026 Job Description Draft):

✅ Design the entity taxonomy for the agent's knowledge domain
✅ Define tiering strategies (what goes in L0 vs L1 vs L2)
✅ Build ingestion pipelines for new knowledge sources
✅ Monitor context freshness and trigger regeneration
✅ Define self-evolution policies (what confidence threshold triggers a merge?)
✅ Design SHACL schemas for knowledge validation
✅ Build provenance dashboards for compliance teams
✅ Run context quality evaluations (is the agent's knowledge accurate?)
✅ Tune conflict resolution logic (what happens when two sources disagree?)
✅ Instrument context hit/miss rates and token usage per query

The last point deserves emphasis. Context engineering has metrics. You can measure L1 cache hit rate (how often does the L1 content you loaded actually get referenced?), knowledge staleness (how often is the agent corrected by a human because its L2 was out of date?), and evolution precision (what percentage of auto-merged knowledge survives the next manual review?). These are engineering metrics, not vibe metrics.

The tooling is arriving to match: the viking status

command shown earlier, Semantica's audit dashboard, and the broader class of "agent observability" tools emerging in mid-2026 are all building toward the same vision — a production control plane for agent context, as rigorous as your database SLOs.

The 24% agent you shipped last quarter isn't a model problem. It's a context problem.

Context engineering for AI agents is the recognition that production agents need the same infrastructure investment we've always given to data: schema design, tiered storage, indexing strategies, provenance tracking, and operational observability. The model handles the reasoning. Your job is to ensure it reasons over the right information, at the right granularity, at the right cost.

The results speak for themselves: 24% → 82% accuracy on long-context memory tasks. 326× latency improvement on multi-hop retrieval. 91% token cost reduction. Double-digit percentage point improvements on production task completion benchmarks. These aren't benchmark games — the VikingMem paper's acceptance at VLDB 2026 signals that the top database research community agrees this is a serious systems problem deserving serious systems solutions.

viking init

on your current agent project: pip install openviking

viking evolve

to your turn completion is the highest-ROI single changeThe shift from prompt engineering to context engineering isn't just a new buzzword — it's the recognition that building production AI systems is a data engineering problem as much as it is an ML problem. The engineers who build that infrastructure in 2026 will be the ones who define what AI agents can actually do in 2028.

Enjoyed this deep dive? Follow me for more posts on AI systems engineering, agent architecture, and the infrastructure layer that makes production AI actually work. Drop questions or push back in the comments — especially if you've run your own context engineering experiments with different results.

References & Further Reading