Graph Engineering: The Missing Layer Between Data and Reasoning

TrueFoundry's working definition of graph engineering—designing the topology of an AI system as an explicit, versioned artifact—addresses a gap that LangChain's 2026 State of Agent Engineering report ties to more than 60% of production agent incidents from state management failures. The field has emerged from the convergence of LangChain's findings, Anthropic's research system, Klarna's assistant, Uber's migration tooling, and LinkedIn's recruiter flows, with a 2026 analysis of Claude Code finding only about 1.6% of its codebase is AI decision logic. Graph engineering structures what a system is—its members, mandates, and message paths—distinct from knowledge graph engineering, which structures what it knows.

A few months back I watched a “simple” customer support agent send the same follow-up email to a customer four times in one afternoon, because two parts of the system both thought they were responsible for closing out the conversation, and neither one checked what the other had already sent. Nothing catastrophic happened. Just an annoyed customer and an awkward Slack thread. But the postmortem was still a mess, because nobody could answer a basic question: what was this system supposed to do at that step? The prompts were fine. The model was fine. The actual problem was that the “system” was a pile of scripts, a couple of cron jobs, and a webhook, and no two engineers on the team had the same picture in their head of how those pieces connected. That is the gap graph engineering is trying to close. And in the last few months it went from something a handful of teams did quietly to something with a name, a hype cycle, and a lot of arguing on X. Let me try to give you the useful version. Graph engineering is designing thetopologyof an AI system as an explicit, versioned artifact instead of letting it emerge from whatever code you happened to write. You decide: TrueFoundry’s working definition https://www.truefoundry.com/blog/graph-engineering-enterprise-guide puts it well: graph engineering designs the topology, loop engineering designs how each agentic node executes. They compose. A graph of unreliable loops is an org chart of unreliable employees. Great loops with accidental topology is coordination failure at scale. One disambiguation before anything else, because search engines have made a mess of this. This is not knowledge graph engineering. Knowledge graphs structure what your system knows : entities, relationships, triples, GraphRAG. Graph engineering in the 2026 sense structures what your system is : its members, their mandates, and their message paths. You will probably use both. They answer different questions. Three things converged. LangChain’s 2026 State of Agent Engineering report ties more than 60% of production agent incidents to state management failures to agents losing context mid-workflow, repeating steps, crashing with no way to recover. A nthropic’s research system, Klarna’s assistant, Uber’s migration tooling, LinkedIn’s recruiter flows. Real traffic, real money. One 2026 analysis of Claude Code’s design space found only about 1.6% of the codebase is AI decision logic. The rest is context management, tool routing, and recovery. Once you internalise that, treating the wiring as a first-class engineering artifact stops sounding like a buzzword. AI maturity evolves from crafting prompts to engineering coordinated systems of agents, tools, workflows, and governance. Graph Engineering treats an AI system as a graph of interconnected nodes, edges, and shared state rather than as a collection of isolated prompts. You’ve probably experimented with ChatGPT or Claude. You write a prompt, get an answer back, tweak the prompt, try again. That’s prompt engineering. It works great until it doesn’t. The problem arrives quietly. Your task grows from “write a summary” to “write a summary, pull in recent data, check it against company policy, and get approval before sending it to the client.” Now a single prompt can’t do the job. You need multiple AI agents doing different things, reading from shared information, and talking to each other. Suddenly you’re not writing prompts anymore. You’re building a system. This progression from prompt engineering to graph engineering is what the infographic is actually about. But it’s worth understanding why each stage exists and what breaks down when you try to skip one. Write better instructions, get better outputs. This works for tasks where the answer fits in one response. A summary, a translation, a first draft of an email. You control the prompts. The limitation is obvious: real work isn’t that simple. The system now needs to remember things. Previous conversations, documents, customer history. You provide the right information as context. The model becomes more useful because it can see the full picture. Problem: context gets out of hand. Too many documents and the system gets confused. Too few and it misses critical details. Figuring out what context to give when becomes a job itself. Instead of one back-and-forth, you give the agent a loop: Plan what to do, execute an action, look at the result, evaluate whether it worked, try again if needed. This makes a single agent much smarter. It can break down complex tasks, iterate, and fix its own mistakes. The catch: you’ve optimized one agent. When your system needs five agents doing different things one researching, one analyzing, one recommending, one checking , loops don’t help with the coordination problem. Each agent might be great at its job, but they’re still isolated. This is where things get real. You stop thinking about prompts or even individual agents. You think about the system as a graph. Agents are nodes. Each node has a role Planner, Researcher, Reviewer, Executor . Edges are the connections between them, defining who talks to whom and what they send. State is the shared memory that all agents read from and write to. Tools are the outside systems they can call databases, web search, CRM, email . The Planner agent receives the ticket and decides what needs to happen. It fans out requests to three other agents in parallel: Research digs into the customer’s account history, Data pulls up relevant metrics, Strategy considers solutions. These three run at the same time, not in sequence. Their results feed into a Reviewer , which checks everything against company policy. If something violates a rule or requires judgment, it gets sent to a human for approval. The human can edit the response or send it back for changes. Once approved, an Executor carries out the decision . The customer gets their answer. Throughout this entire process, there’s a shared state : a place where every agent can see the conversation history, intermediate results, decisions made so far, and the rules that apply. Without this shared state, the agents would be fumbling in the dark. With it, they act like one coherent system. Below that sits a tooling layer. Web search, databases, analytics tools, a CRM, email. Any agent that needs to look something up or trigger an action goes here. And wrapping around everything: observability and governance that never turns off. Logs of what happened, metrics on performance, cost tracking, safety checks, alerts if something goes wrong. Not bolted on at the end. Running the entire time. Most AI setups fail at scale because they’re built on the assumption that one model can handle everything. When you move to graph engineering, you’re admitting that’s not true. One agent is dumb about things outside its specialty. Multiple agents are only as good as their coordination. The graph is your answer to coordination. It forces you to think about: What does each agent actually need to know to do its job? What information needs to be shared? Where do humans need to stay in control? How do you know if something went wrong? What happens if an agent makes a bad call? You can’t bolt these questions on afterward. They have to be in the architecture from day one. A well-designed graph scales. You can add new agents, new tools, new rules without breaking everything else. A badly designed one collapses the moment you add complexity. Once you’re thinking in graphs, you’re no longer prototyping. You’re building something that actually works in the real world. The best reply in that thread was some version of “congrats, you reinvented LangGraph.” That is mostly correct and worth saying out loud. LangGraph shipped StateGraph, nodes, edges, and shared state well before anyone called it graph engineering. Microsoft's AutoGen, Google's ADK, and CrewAI all got there in their own way. Go back further and you find Airflow DAGs, the actor model, contract-net protocols from 1980, and org charts. So this is a naming event , not an invention. Something that had been assembling for decades became discussable as one thing. That still matters. Naming events are when tooling, hiring, and interview questions start to form. Just do not let anyone sell you a paradigm shift when what they mean is “please read the docs.” State is the contract between every node. If it is not in the state schema, it does not exist to the rest of the graph. The subtle part is reducers . When two nodes run in parallel and both write to findings, what happens? Last write wins? Append? You have to say. A reducer is just the merge function. A node is a function that reads state and returns a partial update. It does not have to be an LLM. My strongest opinion in this whole post: if a node does not need a model, do not use one. Routers, validators, formatters, and joins should be boring Python. Every model call you remove is latency, cost, and a class of failure gone. Static edges are “always go here next.” Conditional edges are “look at the state, then decide.” Conditional edges are where your business logic lives, and they are the thing you should be able to point a compliance person at. After each step, state gets written to a store keyed by a thread id. This buys you pause and resume, crash recovery, and time-travel debugging. Without a stable thread id, none of it works. A node can stop mid-execution, save everything, and wait for a human. Approve, edit, or reject, then resume. In LangGraph this is interrupt plus Command resume=... . Useful distinction from the July discourse. The org graph is the static structure you designed: who exists and who may talk to whom. The work graph is what actually ran on a given request, including dynamically spawned subtasks. They are not the same, and most of the interesting engineering is at the seam. Examples use LangGraph, since it is the most widely deployed of these. The ideas port to ADK or Agent Framework with different syntax. pip install -U langgraph langchain python from typing import Annotated, Literal, TypedDictfrom operator import addfrom langgraph.graph import StateGraph, START, END The state schema is the contract. Anything not here is invisible to the rest of the graph.class TicketState TypedDict : ticket: str category: str Annotated + reducer tells LangGraph how to merge writes from nodes that may run in parallel. Here: append. notes: Annotated list str , add def classify state: TicketState - dict: In production this is an LLM call with structured output. text = state "ticket" .lower category = "refund" if "refund" in text else "general" return {"category": category, "notes": f"classified: {category}" }def handle refund state: TicketState - dict: return {"notes": "pulled order history, drafted refund" }def handle general state: TicketState - dict: return {"notes": "answered from the help centre" } The router needs no intelligence, so it gets no model.def route state: TicketState - Literal "refund", "general" : return "refund" if state "category" == "refund" else "general"builder = StateGraph TicketState builder.add node "classify", classify builder.add node "refund", handle refund builder.add node "general", handle general builder.add edge START, "classify" builder.add conditional edges "classify", route the decision pointbuilder.add edge "refund", END builder.add edge "general", END graph = builder.compile print graph.invoke {"ticket": "I want a refund", "category": "", "notes": } Small thing that pays off: graph.get graph .draw mermaid gives you a diagram straight from the code. Paste it in the PR. Reviewers who would never read the orchestration file will happily argue about a picture. This is the orchestrator-worker pattern. A planner splits work, workers run in parallel with their own context windows, a synthesiser merges. python from langgraph.types import SendMAX BRANCHES = 5 cap in code, never in a promptdef plan state: ResearchState - dict: subtopics = decompose state "question" :MAX BRANCHES return {"subtopics": subtopics}def fan out state: ResearchState : One Send per subtopic = one parallel branch, each receiving only the slice of state it needs. return Send "research one", {"subtopic": s} for s in state "subtopics" def research one state: dict - dict: finding = search and summarise state "subtopic" return {"findings": finding } the reducer merges these backbuilder.add conditional edges "plan", fan out, "research one" builder.add edge "research one", "synthesise" Lesson learned the expensive way: hard-ban recursive spawning in the orchestration layer. If a subagent can spawn subagents, you have built a fork bomb with a credit card. Asking it nicely in the system prompt is not a control. python from langgraph.checkpoint.postgres import PostgresSaverfrom langgraph.types import interrupt, Commanddef approve refund state: TicketState - dict: Execution halts here and state is persisted. The process can now die without losing anything. decision = interrupt {"action": "issue refund", "amount": state "amount" } if decision = "approve": return {"notes": "human rejected the refund" } issue refund state "order id" side effect, only past the gate return {"notes": "refund issued" }with PostgresSaver.from conn string DB URL as cp: graph = builder.compile checkpointer=cp cfg = {"configurable": {"thread id": f"ticket-{ticket id}"}} stable id graph.invoke {"ticket": "...", "category": "", "notes": }, cfg ...hours later, different process, same thread id graph.invoke Command resume="approve" , cfg Two traps here, both of which have bitten me: Anthropic’s Research feature is the clearest published case. A lead agent plans, spawns three to five subagents in parallel, each with its own context window, and a separate citation agent attributes claims at the end. On their internal eval it beat single-agent Claude Opus 4 by 90.2% . It also burned roughly 15x the tokens of a normal chat , and their variance analysis found token usage alone explained about 80% of the performance difference. Read that as: the graph won because it bought more parallel thinking, not because agents are magic. Klarna runs its assistant on LangGraph and LangSmith across 85 million users, reporting an 80% cut in average resolution time and handling roughly two thirds of support chats. Uber’s developer platform built an agent network for large-scale code migration and unit test generation, reporting about 21,000 developer hours saved. LinkedIn rebuilt its internal GenAI stack on LangChain and LangGraph, with a hierarchical agent system behind its recruiter product and a text-to-SQL flow for data analysts. Now the counterweight, because it is just as important. Cognition, the Devin team, published Don’t Build Multi-Agents arguing that in practice, agents running in parallel produce fragile systems: decision-making gets dispersed, context is not shared thoroughly, and every action carries implicit decisions that then conflict. You end up adding a reconciliation step to fix problems your own architecture created. Their sharpened principle is worth memorising: extra agents are fine for reading and analysing, but the writes should stay single-threaded. Academic work backs the caution. The MAST paper arXiv:2503.13657 https://arxiv.org/abs/2503.13657 catalogued 14 failure modes across 1,600+ annotated multi-agent traces, and most of them are coordination failures, not model failures. The synthesis I have landed on: fan out for breadth, stay single-threaded for coherence. Independent research directions parallelise beautifully. Editing a codebase does not. The skills that actually transfer: Career advice, bluntly: do not put “graph engineer” on your CV. Put the outcomes. Say you cut agent incident rate by isolating state, or shipped a human-approval gate that let legal sign off on an autonomous workflow. The label will churn. The capability will not. Checkpoints are not durable execution. This is the critique I most agree with. A checkpointer gives you a save point. It does not give you failure detection, automatic resumption, or duplicate-execution prevention. If your process dies, nothing notices until a human does. At a hundred concurrent workflows that is a nuisance. At ten thousand it is an outage. You are still the orchestrator. Cost is topological. Adding one node can multiply spend if it sits inside a fan-out that retries. Debugging is genuinely harder. Non-determinism plus concurrency plus model drift. Time-travel replay helps a lot. Use it. Prompt injection scales with the graph. Untrusted content entering one node can propagate along every edge that node touches. Guardrails at the gateway reduce this. They do not solve it. The vocabulary will churn. “Org graph” and “work graph” may not survive the year. Learn the primitives, stay loose on the nouns. If you take one thing from this: agents are not functions, they are graphs of state transitions. The sooner your code admits that, the sooner your 4am pages stop. Have you shipped a multi-agent graph to production? I want to hear what broke. Drop it in the comments or send it over, and I will fold the good ones into a follow-up on failure patterns. Graph Engineering: The Missing Layer Between Data and Reasoning https://pub.towardsai.net/graph-engineering-9986f8d5bdbd was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.