Using Neo4j Change Data Capture to Orchestrate Agents

Neo4j's Change Data Capture API, available in AuraDB Business Critical and self-managed Neo4j Graph Database Enterprise or Infinigraph Editions, lets teams orchestrate multi-agent pipelines by treating graph writes as events instead of running a separate message broker. The db.cdc.query procedure returns an ordered stream of change records since a cursor position, with selectors such as {"operation": "u", "labels": ["Task"], "changesTo": ["status"]} filtering for specific transitions like a task moving from PENDING to ACTIVE, and ORDER BY txId, seq preserving commit order. Failed events become nodes linked to the task and the agent that could not handle them, keeping the dead-letter triage queryable in Cypher alongside operational state and the audit trail.

Using Neo4j Change Data Capture to Orchestrate Agents 9 min read You’ve built a multi-agent orchestration pipeline. One agent finishes its work and writes a result. The next agent should pick that result up and run. Something has to notice the first agent finished and wake the second one. The usual answer is to add a message broker: publish an event to Kafka, fire a webhook, or stand up a queue. Now you’re running a second system next to your database, keeping two stores in sync, and stitching an audit trail that lives in a separate system to the operational state. If your agents already keep their state in Neo4j, there’s a simpler shape. The graph write is the event. Change Data Capture CDC turns every committed change into an ordered stream you can read, so a task moving from PENDING to ACTIVE becomes the signal that dispatches the next agent. State, events, governance decisions, and the audit trail stay connected in the same graph, queryable with Cypher. CDC is a capability of AuraDB Business Critical https://neo4j.com/product/auradb/?utm campaign=change-data-capture-to-orchestrate-agents/ and self-managed Neo4j Graph Database Enterprise or Infinigraph Editions . If you’re on AuraDB Professional today, it’s one of the reasons teams move up to Business Critical. This post walks through the idea and the trade-offs, drawn from how we run the pattern in production. Because CDC lives on a Business Critical database, this is an architecture piece rather than a spin-it-up-in-two-minutes tutorial. Here’s the whole system at a glance. The graph is the system of record for task state and agent handoffs. CDC is the layer that tells you the moment any of that state changes, so you can act on it. In production the worker is an LLM agent call, and the write-back it produces is itself a change record, which wakes the next agent in the chain. One payoff up front, so you can see where this goes. When an event can’t be handled bad payload, agent down, retries exhausted , most stacks drop it into a dead-letter queue you inspect through a separate tool. In a graph, that failed event becomes a node with relationships to the task it belonged to and the agent that couldn’t take it. You triage it in Cypher, right next to everything else. We’ll come back to it; for now, just note that the failures stay queryable too. What CDC Unlocks Neo4j’s Change Data Capture API https://neo4j.com/docs/cdc/current/?utm campaign=change-data-capture-to-orchestrate-agents/ db.cdc.query returns an ordered stream of change records since a cursor position: every node creation, property update, and relationship edit. Here’s the real read, condensed from the production consumer: CDCSessionManager/consumer.py: query cdc rows rows = self.connection.execute query """ CALL db.cdc.query $cursor, $selectors YIELD id, txId, seq, metadata, event RETURN id, txId, seq, metadata, event ORDER BY txId, seq """, {"cursor": cursor, "selectors": self.config.selectors}, return dict row for row in rows You poll in a loop and advance the cursor on each commit; ORDER BY txId, seq keeps events in commit order. Selectors are maps that filter by label and changed property, so you subscribe only to the transitions you care about. For example, {"operation": "u", "labels": "Task" , "changesTo": "status" } asks for Task status updates and nothing else. In production the call also falls back to an unfiltered read if a selector query fails and no runtime scope is pinned. Each record carries the transaction that produced it and the before-and-after state of the changed entity. A task moving from PENDING to ACTIVE comes back like this, in Aura’s captureMode=DIFF shape: { "id": "BdFcqCXvYIVpAAAAAAAAAAE=", "txId": 43, "seq": 0, "metadata": { "executingUser": "neo4j", "captureMode": "DIFF", "txStartTime": "2026-09-02T17:24:00.412Z", "txCommitTime": "2026-09-02T17:24:00.463Z" }, "event": { "elementId": "4:9b2c1f8a:117", "eventType": "n", "operation": "u", "labels": "Task" , "keys": {"Task": {"task id": "TASK 001"} }, "state": { "before": { "properties": { "status": "PENDING" } }, "after": { "properties": { "status": "ACTIVE", "updated at": "2026-09-02T17:24:00.463Z" } } } } } The id is your cursor for the next poll. txCommitTime tells you when the change committed, keys carries the node’s identity nested under its label, and state.before/state.after tell you exactly what changed. That’s enough to route the event: read labels and state.after.properties.status , and act when a task moves PENDING → ACTIVE . In production, that action is a live agent call. The consumer sees the transition and hands the task to an agent through any LLM API. The agent reads its task from the graph, does its work, and writes new transitions back to the same graph. Those writes are themselves change records, so the next poll picks them up and wakes the next agent in the chain. What You Gain This is the heart of the pattern. Four things come for free once the graph is both your state store and your event source. Your state and your events are the same commit. With a separate broker, a publish can succeed after the database write fails, or the write can land while the publish is lost. You end up reconciling two systems that disagree. With CDC, the change record comes from the committed transaction. If the Neo4j write commits, the event exists. If it rolls back, so does the event. You skip the outbox table and the two-phase dance. Everything lives on one query surface. Task state, agent handoffs, event history, and governance decisions sit in the same graph. You can answer “which tasks have been running for over an hour, and who’s on them” in a single Cypher query, because the task and its assignee are connected nodes rather than rows scattered across a database, a broker, and a log aggregator. php MATCH t:Task {status: 'ACTIVE'} - :ASSIGNED TO - a:Agent WHERE t.updated at < datetime - duration 'PT1H' RETURN t.task id, a.name, t.updated at ORDER BY t.updated at Routing is a selector. Adding a new event type means adding a label and extending a selector. You don’t provision a topic, wire a consumer group, or redeploy a broker config. The example above subscribes to Task status changes; add Handoff to the selector and you’re consuming those too. Failures become first-class nodes. This is the one that surprised me most. When an event can’t be dispatched, you write it back to the graph: MERGE h:Handoff {handoff id: $id} SET h.status = 'DLQ', h.error = $reason, h.source event = $cursor That dead-letter record now has relationships to the task that failed and the agent that couldn’t handle it. You query it, route it to a reviewer, attach notes, and mark it resolved, all in Cypher. In our production system, poisoned events become Handoff nodes routed to a human-review role for triage. The dead-letter queue lives in the graph, alongside everything else you’d query. Replay comes with the same model: point the consumer at an earlier cursor position and it reprocesses from there, as long as that position is still inside the readable window. That window has limits, which brings us to the trade-offs. The Honest Trade-offs Two things to know before you reach for this. It has a latency floor. CDC detection is cursor polling, so it has a floor of roughly 750 ms in the hot path: lag = Bolt write RTT ~10–50 ms + polling interval ~750 ms hot + Bolt read RTT ~10–50 ms Measured on a real AuraDB deployment: | Case | Observed lag write → consumer detect | | Hot recent activity | 750–900 ms | | Idle no events for 5 s | up to 15 s adaptive backoff | | Error recovery | 5–60 s exponential backoff | The idle drift is deliberate. Backing off when there’s nothing to process keeps AuraDB load near zero, and the next poll fires within ~750 ms once work arrives. In a chained agent workflow, each hop adds one polling floor: a three-step pipeline is three transitions, roughly 2.4 s of detection latency end-to-end before the agents’ own work. It reads within a bounded window Aura’s transaction-log rotation and disk limits bound how far back a consumer can read. A saved cursor doesn’t survive the rotation of the transaction log that held its records. Four things follow: - Your poller has to keep pace under heavy writes. Monitor its lag and treat a growing backlog as an operational warning. - A single-poller lease prevents duplicate processing. It does not protect against rotation-overrun loss. - If the poller crashes and stays down until Aura rotates past its cursor, the missed records are gone from the readable window. - When that gap gets expensive, feed CDC into Kafka for durable retention. That last point is the graduation path, and it deserves its own section. When to graduate to Kafka Start with CDC directly when Neo4j is your event source, one consumer lane fits the workload, and ~750 ms detection is fine. You get a small operational footprint and keep state, events, and audit together. Add Kafka or Temporal when you outgrow that envelope: | You need | Reach for | | Sub-100 ms reactions | Kafka same-AZ P99 ≈ 5–20 ms | | Independent fan-out to many consumers | Kafka consumer groups | | Very high sustained throughput | Kafka | | Retention beyond Aura’s log window | Kafka as the durable layer | | Durable timers, sagas, signals | Temporal | Think of it as a graduation. Neo4j stays the source of graph changes; CDC feeds Kafka, and Kafka becomes the durable distribution layer. Temporal works the same way alongside it: when you need durable timers or saga coordination, Temporal runs the workflow while the graph keeps the state. Here’s how the direct-CDC approach sits next to the alternatives. Read the top two rows before the latency rows, because that’s where the real difference is: | | CDC polling | Kafka | Temporal | | State and events live in | One Neo4j graph | Broker, separate from state | Temporal’s own store | | Governance + audit trail | Same graph, queryable in Cypher | Wire it yourself | Workflow history | | Extra infrastructure | None | Broker cluster | Temporal server + DB | | Event latency P99 | ~800 ms | ~5–20 ms | ~100 ms | | Multi-hop pipeline 3+ transitions | ~2.4 s | ~60 ms | ~300 ms | | Fan-out | Single cursor lane | Consumer groups | Native | | Replay | Cursor rewind, within window | Offset reset | Workflow history | The direct-CDC column keeps your state, the event that moved it, the review that approved it, and the audit trail in one connected dataset. The other three columns are faster on the wire, and each keeps operational state in a second system you run and reconcile. That’s the trade: milliseconds for a single store where the whole story of a task is one Cypher query away . Where this comes from This pattern runs in production as the CDCSessionManager behind Aura OS, my personal internal multi-agent system. Two small pieces carry most of the weight, and both live on the same graph as the data. One poller per lane. A short lease guarantees a single active poller per cursor, so two workers never process the same event. Whoever wins the SET owns the lane until the lease expires: // cursor store.py: acquire lease MATCH c:CDCConsumerCursor { consumer id: $consumer id, database name: $database name, selector profile: $selector profile } WHERE coalesce c.status, 'ACTIVE' < 'BROKEN' AND c.lease owner IS NULL OR c.lease owner = $lease owner OR c.lease expires at IS NULL OR c.lease expires at < datetime $now iso SET c.lease owner = $lease owner, c.lease expires at = datetime $lease expires iso , c.last poll at = datetime $now iso RETURN c.consumer id AS consumer id Idempotent dispatch. Every handler effect is keyed to a deterministic receipt, so replaying a change record can’t fire the same side effect twice: python dedupe.py: build receipt id def build receipt id , consumer id, cdc change id, handler id, entity id, action key : raw = "|".join consumer id, cdc change id, handler id, entity id, action key return hashlib.sha256 raw.encode "utf-8" .hexdigest Where to go next? Neo4j CDC is available on Aura Business Critical Tier https://neo4j.com/product/auradb/?utm campaign=change-data-capture-to-orchestrate-agents/ and on a self-managed Neo4j Enterprise Edition License. If Neo4j already holds your agent system’s state, it’s the piece that turns the graph into your event bus, with no separate broker to run and reconcile. - The Change Data Capture documentation https://neo4j.com/docs/cdc/current/?utm campaign=change-data-capture-to-orchestrate-agents/ covers selectors, cursors, and capture modes. - Start a Free Trial of Aura Professional https://neo4j.com/product/auradb/?utm campaign=change-data-capture-to-orchestrate-agents/ to model your graph, then move up to Business Critical to turn CDC on.