We built Saturn, an agent that answers questions about a company's decisions, people, systems, and documents by querying a FalkorDB graph.
Questions like "Who owns this system?", "Why did we make this decision?", and "Is this information still current?" sound simple. They become harder when the answer is spread across documents, conversations, and decisions made months apart.
Saturn connects that information instead of treating every document as an isolated piece of text. Each answer comes with the IDs of the graph nodes it used, and, where the question involves a chain, the path between them.
We tested it with 70 questions. The graph plus LLM setup scored 60/70, our deterministic router scored 59/70, and a simple lexical retrieval baseline scored 20/70.
These are results from our own smoke test, not a leaderboard. The baseline is simple: it can't refuse a question or show a path, and our keyword-based scoring has limitations. We'll explain what we tested, what worked, and what still needs improvement.
The problem we wanted to solve
Imagine you're working on a product with a team of engineers.
In December 2025, someone documents that the mobile API should allow 50 requests per second per user. An engineer makes a decision based on that policy, and the team moves on.
Then September 2026 arrives. A new policy raises the limit to 200 requests per second.
The old decision is still sitting in your documentation. Nobody has updated it.
Now someone asks an AI assistant, "What is our mobile API rate limit?"
A retrieval-based system might find the old document, notice that it matches the question, and return 50 requests per second. The document isn't necessarily wrong about what the policy used to be. The problem is that the system doesn't know the policy has changed.
We wanted Saturn to handle exactly this kind of situation.
We used a fictional company called NovaTech to test the idea. All company data in this example is synthetic.
A graph showing Rate Limiting Policy v1 being superseded by v2, which took effect on September 15, 2026. Two decisions based on the old policy are flagged as potentially stale.
Synthetic seed data. A single SUPERSEDES relationship connects the new policy to the old one, allowing Saturn to find decisions that depend on outdated information. The second flagged decision is a randomly generated filler record.
Instead of storing both policy documents without a connection, Saturn links them with a SUPERSEDES relationship. We can follow that connection to find decisions based on the old policy and flag them for review.
In our demo, the new policy took effect on September 15, 2026, and our detector first ran on October 18. That's a 33-day gap between when the change became effective and when we detected it.
We store both dates. Knowing that something is outdated is useful. Knowing when it became outdated, and when your system noticed, is even more useful.
What we built
We called the project Saturn. The repository is still named provenance.
It's our entry for Graph Hacks Track 3, Company Brain.
Saturn is designed to answer four kinds of questions:
The goal isn't just to generate an answer. Saturn also returns the IDs of the graph nodes it used and the path connecting them where applicable. If the graph has no record of the information being requested, Saturn should say so rather than inventing an answer.
We use one FalkorDB instance with three isolated graphs:
One query layer sits behind a REST API, a Next.js interface, and an MCP server that exposes 14 tools. Compatible clients such as Claude Desktop or Cursor can query the same underlying functionality.
Why we chose FalkorDB
We could have stored this information in PostgreSQL and retrieved relevant documents. A relational database can handle parts of the problem: counting contributors is a straightforward GROUP BY, and recursive queries can follow supersession chains.
But most of Saturn's questions are about relationships.
A decision connects to the person who made it, the documents behind it, the systems it affects, and sometimes another decision that it replaced. Finding an answer can mean following several of these connections.
FalkorDB fit this use case. Variable-length paths and shortestPath are available directly in Cypher. Relationships can carry properties, giving us a natural way to represent valid_from and valid_to. Multi-graph support also lets us isolate workspaces without separate deployments.
We added tests to check that data from one workspace doesn't leak into another.
The important question for us was whether the graph did something useful beyond holding the data. In Saturn, it powers decision provenance, stale-information detection, historical lookups, and bus-factor analysis.
One design decision shaped the implementation: the language model never writes the database query.
Every tool uses a fixed Cypher query. A keyword-based router selects the tool by default, or the LLM can select a tool when an API key is configured. The model can also extract entities from new text and help phrase the answer.
The query itself remains controlled by our application.
How Saturn works
Saturn uses three separate workspaces in one FalkorDB instance. The web UI, REST API, and MCP server all go through the same query layer, so they access the graph consistently. Every answer includes node IDs that you can check directly in the graph.
Our graph has nine node types: Person, Team, System, Project, Decision, Document, Ticket, Message, and ContradictionCandidate. It has 13 relationship types.
Edges carry valid_from, valid_to, ingested_at, and source_id. Together, these properties help us trace facts to their sources and understand when they were valid.
Three queries illustrate most of what Saturn does.
Why was this decision made?
Here's an abridged Cypher query:
MATCH (d:Decision {id: $decision_id})
OPTIONAL MATCH (d)-[:MADE_BY]->(maker:Person)
OPTIONAL MATCH (d)-[:BASED_ON]->(source)
OPTIONAL MATCH (d)-[:SUPERSEDES]->(old:Decision)
OPTIONAL MATCH (source)<-[:RESOLVES]-(ticket:Ticket)
OPTIONAL MATCH (ticket)-[:DISCUSSED_IN]->(msg:Message)
OPTIONAL MATCH (originator:Person)-[:AUTHORED]->(msg)
Starting from a decision, the query follows its connections to the person who made it, its source documents, the decision it replaced, and, where the data is available, the ticket and discussion where the idea originated.
What just went stale?
MATCH (new_doc:Document {id: $new_doc_id})-[:SUPERSEDES|CONTRADICTS]->(old_doc:Document)
MATCH (d:Decision)-[:BASED_ON]->(old_doc)
WHERE d.status <> 'superseded'
OPTIONAL MATCH (d)-[:AFFECTS]->(s:System)<-[:AFFECTS]-(downstream:Decision)
WHERE downstream.id <> d.id AND downstream.status <> 'superseded'
The first part finds decisions that depend on an older document. The second looks for other decisions connected to the same system.
Those connections help us identify decisions that might need review. Sharing a connection to the same system doesn't prove that one decision depends on another, so we treat this as a review signal rather than a confirmed dependency.
What was true on a particular date?
MATCH (d:Decision)
WHERE coalesce(d.valid_from, '') <= $as_of_date AND d.status <> 'draft'
Cypher selects decisions that had started by the requested date. Python then calculates their status at that point in time, treating the start of a superseding decision as the end of the old decision's validity.
The use of coalesce is deliberate. We'll explain why in the lessons section.
We also wanted contradictions to be handled carefully. If two documents appear to disagree, Saturn creates a ContradictionCandidate node. A person must confirm the contradiction before a real CONTRADICTS relationship is created.
We didn't want an automated process silently rewriting the graph just because two pieces of text looked inconsistent.
One question, end to end
Consider this question:
"What did we believe about JWT authentication for AuthService as of May 1, 2025?"
Here's what happens:
as_of tool. No LLM is involved in this run.
A provenance receipt showing a question, an as_of answer, and three cited graph node IDs: decision_use_jwt, doc_adr_jwt_original, and person_priya.
This is a rendering of the API response, not a UI screenshot. Python calculates the ACTIVE status from the validity interval, and each cited node can be checked directly in FalkorDB.
In this example, the decision's validity window runs from April 23, 2025, to August 16, 2026.
Ask the same question as of September 1, 2026, and the status changes to SUPERSEDED. Nothing about the stored record needs to change between those queries. We're asking about two different points in time, so the answer changes accordingly.
A timeline showing a decision's validity period from April 23, 2025, to August 16, 2026, with different as_of dates returning active or superseded status.
Synthetic data. The decision isn't rewritten to produce different answers. The ACTIVE or SUPERSEDED status is calculated in Python from the validity interval.
Does it beat plain retrieval?
We ran a small benchmark to get an initial idea of how Saturn performs.
We wrote 70 questions across three datasets: 42 about NovaTech, 13 about the FalkorDB repositories, and 15 about PEP records. Fourteen questions deliberately contained false premises, such as asking who authored a PEP that didn't exist in the loaded data.
We compared three approaches:
A bar chart comparing results across 70 questions. Graph + LLM scored 85.7%, the raw router scored 84.3%, and lexical retrieval scored 28.6%.
A single smoke test, not a leaderboard result. Scoring is keyword-based. The datasets include one synthetic workspace and two real datasets: merged GitHub pull requests and PEP records. The lexical baseline cannot refuse a question or provide a graph path, so some differences reflect capabilities that the baseline doesn't have.
| Category | Questions | Graph + LLM | Raw router | Lexical retrieval |
|---|---|---|---|---|
| Decision provenance | 17 | 15/17 | 14/17 | 2/17 |
| Who knows | 18 | 14/18 | 18/18 | 16/18 |
| Staleness | 8 | 5/8 | 4/8 | 0/8 |
| Bus factor | 5 | 5/5 | 5/5 | 0/5 |
| As-of time travel | 4 | 4/4 | 2/4 | 0/4 |
| Buried origin | 2 | 1/2 | 1/2 | 0/2 |
| Connection path | 2 | 2/2 | 1/2 | 0/2 |
| False premise | 14 | 14/14 | 14/14 | 2/14 |
| Overall | 70 | 60/70 | 59/70 | 20/70 |
Note: In decision provenance, 11 of the 17 lexical-baseline cells were unscored and counted as misses.
The numbers are useful, but they need context.
Our baseline is simple. It can't refuse a question when the information doesn't exist, and it can't return the path connecting an answer to its source. A stronger embedding-based retrieval system combined with an LLM would likely do better. We haven't tested one yet.
Our scoring is also keyword-based and fairly generous. An answer can pass if it mentions the correct entity without fully answering the question.
Fourteen baseline cells were left blank and counted as misses. Excluding them, the baseline scores 20/56, or about 36%. Two Graph + LLM cells were also blank; excluding them gives 60/68, or about 88%.
The graph didn't win every category. For "who knows," lexical retrieval scored 16/18, while Graph + LLM scored 14/18. Many of these questions ask for a PEP's author, information that appears directly in a document header. In those cases, ordinary retrieval does a reasonable job.
On the PEP dataset, the LLM path also made four authorship mistakes that the raw router avoided.
Our GitHub ingestion doesn't extract supersession relationships between pull requests, so questions that depend on those missing connections failed in every mode.
We also tested 20 paraphrased questions containing typos and casual wording. Graph + LLM scored 17/20, while the raw router scored 20/20. Eight questions were blind tests written by a separate AI agent. The LLM path scored 5/8, including one miss caused by a 30-second timeout. The router scored 8/8. Given our generous scoring, we treat 20/20 as an upper bound rather than proof that the router is perfect.
We aren't making a speed claim, either. In the paraphrase run, the router averaged 18 ms per answer, while the LLM path took about 3.5 seconds. We haven't separated query execution time from end-to-end latency in the main benchmark.
On this test, the graph path performs well when answers need structure and relationships, but plain retrieval can do better when the answer is already contained in a single document.
What we learned while building it
Store when something became true, not just when you noticed
Our first version recorded when the stale-information check ran. That created a misleading result: a policy could have been outdated for weeks, but the stale flag looked brand new because the system had only just detected it.
We separated stale_effect_at from stale_detected_at and added notice_lag_days. That made the delay visible instead of hiding it inside the data.
Sometimes the right answer is to say you don't know
Before answering, Saturn checks whether the entities in the question exist in the graph. If they don't, it responds with something like "No record of X in the graph."
Both the graph plus LLM path and the raw router declined all 14 false-premise questions in our benchmark. We don't consider that a fair head-to-head comparison against our lexical baseline, because the baseline has no equivalent refusal mechanism. But we do think the design is worth keeping.
Finding a relevant document and establishing that a fact exists are two different things.
Passing tests doesn't mean the database is doing what you expect
We ran into a confusing issue while working on historical queries.
A bare condition like WHERE d.valid_from <= '2025-05-01' matched every decision, including some from 2026. Yet the same comparison inside a RETURN expression behaved correctly.
Wrapping the property in coalesce(d.valid_from, '') fixed the problem.
We suspect the indexed property is involved, but we haven't reduced the issue to a minimal reproduction. We can't say whether it's an engine bug or something we're misunderstanding.
Our tests stayed green throughout. We only caught the problem when an MCP client asked a question whose answer we already knew.
Don't automatically turn every disagreement into a fact
We initially considered automatically flagging documents as contradictory whenever they appeared to disagree.
Instead, we made contradictions candidates that require human confirmation.
A scan of the existing graph initially found nothing. Then we ingested an ADR that reversed an earlier decision, ADR-014 and ADR-017, and the scan flagged the pair.
We preferred that explicit confirmation step over silently creating relationships that might be wrong.
Watch out for shortcuts that only work in demos
We found a hardcoded effective date for one demo decision in our own code. It was the kind of shortcut that makes a demonstration look correct while potentially making a benchmark misleading.
We replaced it with a value derived from the superseding document's valid_from property.
Two smaller issues
Some of our workspaces were empty on one machine. We didn't notice until an MCP client routed a question to them.
We also ran into a configuration issue with Claude Desktop. It didn't accept a direct URL-plus-header configuration for our server, so we used the mcp-remote stdio bridge with an X-API-Key header.
What's next?
There are four things we'd like to improve:
Try it yourself
Live Demo: Try Saturn Live
Repository: GitHub Repository
Benchmark data: bench/benchmark_results.json