{"slug": "does-rag-need-better-retrieval-or-better-relationships", "title": "Does RAG Need Better Retrieval — or Better Relationships?", "summary": "A developer built a controlled experiment testing whether explicitly stated relational context between documents improves retrieval-augmented generation reasoning when retrieval is held constant. Using a synthetic corpus of 254 business documents and 60 multi-hop cases with byte-identical retrieved documents across conditions, typed relationship hints (such as superseded-by or contradicts) raised accuracy from 77% to 94% for DeepSeek and from 77% to 98% for GPT-4o-mini. The author notes the lexical BM25-style baseline had a mean oracle-document recall of 0.67, so results may differ under stronger production retrieval.", "body_md": "I started this experiment for a slightly different reason.\n\nI was playing with a system that could reconstruct context across documents.\n\nNot summarize documents. Not retrieve the most similar chunks.\n\nReconstruct context.\n\nFor example, suppose I have two documents:\n\nIn 2022, the team chose PostgreSQL.\n\nAnd somewhere else:\n\nAfter the migration problems, the team moved to DynamoDB in 2024.\n\nNeither document contains the answer to:\n\nWhy did the database decision change?\n\nThe answer lives in the relationship between them.\n\nThat got me thinking about something else.\n\nIf a system can reliably recover these relationships, could it also use them to create structured training data?\n\nBefore going there, I wanted to know whether the relationships actually mattered.\n\nSo I built a small experiment.\n\nAnd the first result was much stronger than I expected.\n\nThe obvious problem with testing relationships in RAG is retrieval.\n\nIf the experiment gets better answers after adding a graph, what actually improved?\n\nMaybe the graph found documents that retrieval missed.\n\nMaybe it simply retrieved more useful evidence.\n\nMaybe the model got more context.\n\nMaybe the graph structure itself helped.\n\nThose are very different explanations.\n\nI wanted to isolate one of them.\n\nSo I made the retrieved documents identical in both conditions.\n\nNo re-ranking.\n\nNo additional documents.\n\nNo better retrieval.\n\nThe model gets the same documents.\n\nThe only difference is whether it also receives explicit relationships between those documents.\n\nThat became the experiment:\n\n**Does explicitly stated relational context improve reasoning when retrieval is held constant?**\n\nI built a synthetic corpus of 254 business documents.\n\nThe documents cover things like billing, DevOps and HR.\n\nThen I created 60 multi-hop cases with manually defined ground truth:\n\nSome cases require reasoning across several documents. Six of the supersession cases involve four to six documents with unrelated distractors.\n\nI deliberately made some of the relationships hard to recover from individual documents.\n\nFor example, two documents can both be perfectly valid while representing different states of the same decision.\n\nThe important information isn't inside either document.\n\nIt's in:\n\n`A → superseded by → B`\n\nor:\n\n`A ↔ contradicts ↔ B`\n\n`A → belongs to → cluster B`\n\nThere is also a meaningful negative case:\n\n`A → no known relation → B`\n\nThat one turned out to matter more than I expected.\n\nI used a lexical BM25-style top-5 retrieval over the full corpus.\n\nIt isn't a production BM25 implementation. It's a deliberately simple lexical baseline.\n\nMean recall of the oracle documents in the top five was 0.67.\n\n27 of the 60 cases had full recall.\n\n17 had partial recall.\n\n10 had low recall.\n\nSix had zero recall, all of them negative cases.\n\nThis is a real limitation.\n\nIf retrieval misses evidence, a relational hint can potentially compensate for that weakness.\n\nSo I don't want to pretend this experiment answers what happens with a very strong production retrieval system.\n\nBut it gives us something useful:\n\nC and D see **byte-identical retrieved documents**.\n\nThat part is controlled.\n\nThe baseline looks roughly like this:\n\n```\nretrieved documents\n        ↓\n       LLM\n        ↓\n      answer\n```\n\nThe experimental condition adds a small structured block:\n\n```\nretrieved documents\n        +\nexplicit relationships\n        ↓\n       LLM\n        ↓\n      answer\n```\n\nFor example:\n\n```\nkind: superseded\nfrom: database-choice-2022.md\nto: database-choice-2024.md\nreason: migration failure\n```\n\nThe model was told that the knowledge-base engine had detected the relation and that it should use the hint if it was consistent with the documents.\n\nNothing else changed.\n\nI ran the main experiment with two models.\n\n| Model | No relation | Typed relations | \n|---|---|---|\n| DeepSeek | 77% | 94% | \n| GPT-4o-mini | 77% | 98% | \n\nThe gain was:\n\n**+17 percentage points** for DeepSeek.\n\n**+21 points** for GPT-4o-mini. \n\nThat was enough to make me stop and ask a more annoying question:\n\n**What exactly did I measure?**\n\nBecause “adding a graph improves RAG” would be a very easy conclusion.\n\nAnd probably a premature one.\n\nThe relation hints were only around 17 words.\n\nPerhaps the model simply benefited from getting more information.\n\nSo I added a control with approximately the same amount of unrelated text.\n\nParking.\n\nCafeteria.\n\nMeeting rooms.\n\nNo relation labels.\n\nNo document names.\n\nNo similarity scores.\n\nThat condition scored 53%.\n\nThe baseline was 77%.\n\nThe relational condition was 98%.\n\nSo “more tokens” doesn't explain the result.\n\nAt least not by itself.\n\nThe relational information was presented as fields:\n\n```\nkind:\nfrom:\nto:\nreason:\n```\n\nMaybe that formatting alone makes the model perform better.\n\nSo I took the same semantic information and put it into ordinary prose.\n\nSomething like:\n\nNote: B reflects the current state of this decision. A is the previous version.\n\nSame information.\n\nNo structured relation fields.\n\nThat scored 53%.\n\nThe structured version scored 98%.\n\nThis result bothered me.\n\nBecause now I had learned something I hadn't originally set out to test:\n\n**how the relationship is presented may matter almost as much as the relationship itself.**\n\nAnd I had a new confound.\n\nMaybe the model doesn't actually reason over the relationship.\n\nMaybe it just trusts the block because I told it that a “knowledge base engine” generated it.\n\nI'll come back to that.\n\nThis was the test I cared about most.\n\nIf relationships really carry useful information, incorrect relationships should matter.\n\nI corrupted the identifying fields of the relation hints.\n\nAt 20% corruption, accuracy dropped from 98% to 53%.\n\nAt 40%, it was 52%.\n\nThat is a dramatic failure.\n\nBut there is an important detail here.\n\nI did **not** test every corruption level.\n\nI don't know what happens at 5%.\n\nI don't know what happens at 10%.\n\nI don't know whether another corruption mechanism would behave differently.\n\nSo I can't say:\n\n“80% correct relations are useless.”\n\nThe experiment doesn't show that.\n\nIt shows something narrower:\n\nUnder this particular corruption mechanism, introducing 20% incorrect relations almost completely destroyed the observed benefit.\n\nThat distinction matters.\n\nThis was probably the most interesting part of the experiment.\n\nIf relationships help because they connect documents, perhaps we don't need explicit relation types.\n\nMaybe a similarity graph is enough.\n\nSo I created another condition where the model received document-to-document similarity edges instead of typed relations.\n\nThe result improved substantially:\n\n| Model | No hint | Similarity | Typed relation | \n|---|---|---|---|\n| DeepSeek | 76% | 88% | 96% | \n| GPT-4o-mini | 76% | 93% | 98% | \n\nSimilarity clearly helped.\n\nBut it didn't behave like typed relations.\n\nThere was a particularly interesting case.\n\nFor DeepSeek, similarity actually beat typed relations on the link cases:\n\n**79% vs 71%.**\n\nThe likely explanation is still only a hypothesis. The typed hint contained a `reason` field that may have distracted the model. \n\nThe negative cases were different.\n\nSimilarity hurt them.\n\nThe model saw related-looking documents and tended to assume that a relationship existed.\n\nThe typed relation could explicitly say:\n\n```\nkind: none\n```\n\nSimilarity says:\n\nThese things look related.\n\nA typed relation can say:\n\nThese things are related in this specific way.\n\nOr:\n\nThese things are not related.\n\nThose are different pieces of information.\n\nI added three more controls:\n\n**Shuffled relations**\n\nGive each case a relation from another case of the same kind.\n\n**Token-matched text**\n\nGive the model unrelated text with approximately the same length as the relation hint.\n\n**In-document prose**\n\nPut the same semantic relation into ordinary document text.\n\nAll three landed around 52–53%.\n\nThe structured, correct relation condition was 98%.\n\nThe shuffled condition also passed an anti-leakage check: none of the 60 shuffled hints happened to be correct.\n\nAt this point I was reasonably confident that the result wasn't just:\n\n“Give the LLM some extra words that look vaguely useful.”\n\nBut I still wasn't confident about *why* the structured relation worked.\n\nThere is one explanation my controls don't eliminate.\n\nMaybe the model isn't using the relational structure as relational structure.\n\nMaybe it sees:\n\n`Retineo relation: kind=superseded ...`\n\nand interprets that as privileged metadata.\n\nIn other words:\n\n“The system says this is the answer, so I'll trust it.”\n\nThat would produce a large improvement without requiring the model to reconstruct anything from the relationship itself.\n\nThe in-document prose control actually makes this possibility more interesting.\n\nThe two conditions contain the same semantic information.\n\nYet the structured version was dramatically more effective.\n\nSo I can't currently separate:\n\n**relational reasoning**\n\nfrom\n\n**authority/salience effects caused by the representation.**\n\nThat's the next control I want to run.\n\nMy retrieval baseline is weak.\n\nThat's not an incidental detail.\n\nMean top-5 recall was only 0.67, so there is plenty of room for a relational hint to compensate for retrieval failures.\n\nImagine that the model needs documents A, B and C.\n\nThe retriever returns A and B.\n\nThen I give it a relation saying:\n\n```\nA → related_to → C\n```\n\nI haven't really solved the retrieval problem.\n\nI've given the model a shortcut around it.\n\nSo the next experiment is becoming pretty obvious.\n\nI want to vary retrieval depth while keeping the relational condition intact.\n\n```\nshallow retrieval\n       ×\nno relation / relation\n```\n\nthen:\n\n```\ndeeper retrieval\n       ×\nno relation / relation\n```\n\nThe question is not whether deeper retrieval improves the baseline.\n\nOf course it can.\n\nThe interesting question is what happens to the **gap** between the two conditions.\n\nIf:\n\n```\n+21 pp\n  ↓\n+10 pp\n  ↓\n +2 pp\n```\n\nthen the relational hint may mostly be compensating for weak retrieval.\n\nIf the gap remains large as retrieval quality improves, the story gets more interesting.\n\nBut this experiment needs some care. Simply dumping 100 documents into the context changes the amount of text the model has to process, so that would introduce another variable.\n\nI want to separate **more available evidence** from **more text in the prompt**.\n\nHere is the version I'm comfortable with.\n\nOn this synthetic corpus, with this retrieval setup and these two models:\n\n**Correct typed cross-document relations improved accuracy when the retrieved documents were identical.**\n\nThe effect survived controls for extra tokens, shuffled relations, and expressing the same semantic information as ordinary document prose.\n\nSimilarity structure also helped, but typed relations provided additional information in several cases, particularly where relation type or explicit absence mattered.\n\nThat's the observation.\n\nThe explanation is still open.\n\nI don't know yet how much of the effect comes from:\n\nAnd I definitely don't know whether the effect survives real organizational data.\n\nThe corpus is synthetic.\n\nThe retrieval is simplified.\n\nThere are only two models.\n\nThose are real boundaries on the result, not footnotes to hide at the end.\n\nThere was a practical reason I ran the experiment in the first place.\n\nI was thinking about whether a context-reconstruction system could help create structured datasets.\n\nTake a collection of documents.\n\nFind candidate relationships.\n\nClassify them:\n\n```\nsupersedes\ncontradicts\nbelongs_to\nreferences\nnone\n```\n\nThen use those relationships as structured training data.\n\nThat sounds useful.\n\nBut it also creates a nasty circular problem.\n\nIf I use the system to generate relational labels, and then use those labels to demonstrate that relational context helps models, I may just be measuring my own assumptions.\n\nSo I stopped.\n\nBefore thinking about dataset generation, I wanted to know whether the representation itself had a measurable effect.\n\nThat led to this experiment.\n\nAnd now I have a better question than the one I started with.\n\nI don't want to build a bigger graph yet.\n\nI want to try to make this result disappear.\n\nBetter retrieval.\n\nMore evidence.\n\nInteractive retrieval.\n\nDifferent models.\n\nDifferent relational representations.\n\nAnd especially an experiment that removes the “trusted metadata” explanation.\n\nIf better retrieval makes the relational advantage disappear, that's useful.\n\nIf it doesn't, that's useful too.\n\nEither result tells me where to look next.\n\nThe interesting question is no longer:\n\n**Does GraphRAG work?**\n\nThat's too broad to be useful.\n\nThe question I'm interested in is much smaller:\n\n**When an LLM already has the relevant documents, does explicitly representing the relationships between those documents give it information it cannot reliably reconstruct from the documents themselves?**\n\nI don't have the final answer yet.\n\nThat's exactly why I'm running the next experiment.", "url": "https://wpnews.pro/news/does-rag-need-better-retrieval-or-better-relationships", "canonical_source": "https://dev.to/valerykot/does-rag-need-better-retrieval-or-better-relationships-4fkk", "published_at": "2026-09-17 19:17:28+00:00", "updated_at": "2026-09-17 19:53:02.738297+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "natural-language-processing"], "entities": ["DeepSeek", "GPT-4o-mini", "PostgreSQL", "DynamoDB"], "alternates": {"html": "https://wpnews.pro/news/does-rag-need-better-retrieval-or-better-relationships", "markdown": "https://wpnews.pro/news/does-rag-need-better-retrieval-or-better-relationships.md", "text": "https://wpnews.pro/news/does-rag-need-better-retrieval-or-better-relationships.txt", "jsonld": "https://wpnews.pro/news/does-rag-need-better-retrieval-or-better-relationships.jsonld"}}