{"slug": "connecting-coding-agents-to-the-right-documentation-with-a-navigable-context", "title": "Connecting coding agents to the right documentation with a navigable context layer", "summary": "Redis engineers rebuilt the retrieval layer behind an agent interface for redis.io/docs, moving from a single broad documentation entity, RedisioDoc, to multiple entities linked by explicit relationships so coding agents can move from discovery to cited evidence. The V1 schema generated six tools, of which the application exposed four to the agent (get_redisiodoc_by_id, search_redisiodoc_by_text, search_redisiodoc_by_content_embedding_similarity and filter_redisiodoc_by_topic), and the team built a standalone evaluation kit that scores a RetrievalTrace against a golden dataset before the model writes an answer. The team reported stronger retrieval overall from the multi-entity design, at the cost of more tool calls and slower tail latency.", "body_md": "Blog\n\n# Connecting coding agents to the right documentation with a navigable context layer\n\nAsk a question about Redis and the job seems simple: find the right documentation and explain it. **But documentation questions come in many shapes.**\n\n“What does `XACK` do?” is a precise command question. “How should I think about eviction policies?” is a broader concept question. “Why is my stream consumer falling behind?” may span several pages, commands, and ideas.\n\nWe set out to build an agent interface that could answer Redis questions with precise, cited evidence from [redis.io/docs](https://redis.io/docs/latest/). This post follows how we evolved from one broad document entity to multiple entities, linked by explicit relationships that help the agent move from discovery to evidence. Our evaluations revealed stronger retrieval overall, but at the cost of more tool calls and slower tail latency.\n\n## Starting simple\n\nContext Retriever lets developers model their data like an ORM. An “entity” defines a schema for data; Context Retriever exposes retrieval tools that an agent can call. The application still decides which tools to give the agent and how retrieved evidence should be used in the final answer.\n\nFor V1, we started with one entity to represent a documentation page, section, or chunk of text: `RedisioDoc`. It carried a stable ID, title, topic, section, content, vector embedding, and canonical URL.\n\nBelow is the data model for records derived from documentation:\n\nThe above schema generated six tools, but our application only exposed four of them to the agent:\n\n| Generated tool | Exposed to agent? | \n|---|---|\n| get_redisiodoc_by_id | Yes | \n| search_redisiodoc_by_text | Yes | \n| search_redisiodoc_by_content_embedding_similarity | Yes | \n| filter_redisiodoc_by_topic | Yes | \n| filter_redisiodoc_by_subtopic | No | \n| filter_redisiodoc_by_version | No | \n\nThis was a useful baseline. A small toolset made the system easier to inspect while we improved documentation coverage, stable IDs, citations, query handling, and result size.\n\nBut the simple model also flattened several retrieval jobs into one shape. Whether a question needed a page overview, an exact command reference, or a precise passage, the agent searched the same entity and had to sift through the results itself. Our evaluations eventually showed where that approach began to break down.\n\n## When a good answer hid weak retrieval\n\nAt first, we judged the system as a user would.\n\n**Did the answer sound right? Did it cite relevant pages? How long did it take?**\n\nThose questions matter, but they miss a problem. A strong foundation model can turn incomplete evidence into convincing prose, using general knowledge to fill gaps, so a fluent answer can sit on weak retrieval.\n\nThat matters even when the final answer looks fine. If retrieval is weak, the model has to read more irrelevant text, make more calls, and rely more on what it already knows. That costs latency and tokens, and it pushes you toward a larger model to compensate. Good retrieval lets a smaller, faster model produce the same answer from evidence it can cite.\n\nWe therefore began evaluating the evidence, with hard metrics, before the model wrote the answer. This gave us a separate view of retrieval quality: What pages did the agent find? Where did they rank? Which tools led it there?\n\n## Evals over golden data\n\nTo understand whether retrieval was improving, we needed to measure the evidence before the model turned it into an answer. We built a standalone evaluation kit that could test different retrieval agents, MCP servers, APIs, or saved traces against the same documentation corpus.\n\n**[Image placeholder]** Evaluation kit workflow diagram (01 Set up a new retrieval target → 02 Run the shared evaluation). Upload from the source draft. Suggested alt text: “Two-phase evaluation workflow: an agent skill writes a target-specific adapter, then a reusable harness scores the adapter’s RetrievalTrace against a golden dataset.”\n\nThe visual above shows the workflow in two phases.\n\nFirst, we connect a retrieval target to the kit. An agent skill guides this setup by inspecting representative raw outputs and creating a small, target-specific adapter. The adapter knows how to call that system, parse its response, preserve the original result order, and map its source identifiers into stable documentation page IDs. It contains the system-specific integration logic, but no scoring logic.\n\nOnce the adapter is validated, the shared evaluation can run. For each row, the golden dataset supplies:\n\n1. A question.\n2. The primary documentation sources expected to answer it.\n3. Related sources that may also provide useful evidence.\n\nThe questions were divided into four groups that represented different retrieval patterns:\n\n- **Simple lookups** asked for one clear concept or command. For example, “What does`BITCOUNT` do?” expected the`BITCOUNT` command page as its primary source.\n- **Multi-step questions** required evidence from several sources. “How do I create a Bloom filter, add items, and check membership?” expected the`BF.RESERVE` ,`BF.ADD` , and`BF.EXISTS` command pages.\n- **Hard cases** included troubleshooting questions, ambiguous terminology, and questions that crossed several parts of the documentation. For “`XACK` did not reduce stream memory. What should I trim or delete?”, the expected sources included the`XACK` ,`XTRIM` , and`XDEL` command pages and the Streams guide.\n- **Chat-style questions** reflected shorter, production-like prompts. “How do I monitor Redis latency?” expected the Redis latency guide.\n\nThe reusable harness sends the question through the adapter, which invokes the retrieval target. The adapter converts the raw response into a common `RetrievalTrace`: an ordered list of retrieved page IDs, together with the tool calls, errors, and latency recorded during retrieval.\n\nThe harness then removes duplicate pages without changing their original order and compares the ranked results with the expected sources. This lets us measure both whether the right evidence appeared and how early it appeared. Each run produces a detailed `trace.jsonl` for inspecting individual questions and a `report.md` containing metrics such as nDCG, recall, and hit rate.\n\nSeparating the adapter from the harness made the kit reusable. To evaluate another agent or MCP over the same corpus, we write a new adapter while keeping the golden dataset and scoring process unchanged. It also keeps two questions separate: did the agent retrieve the right evidence, and did the model use that evidence to produce a good answer?\n\n## The problem with one broad “document”\n\nThe trace pointed to a problem with the shape of our context model.\n\nOne trace made the problem clear. When asked, “What does `BF.RESERVE` do?”, V1 searched the full documentation collection four times. Its first results included Kubernetes and REST API pages that happened to contain words such as “reserve” or “reservation.” The Bloom filter guide appeared only near the bottom of the first ten results, while the `BF.RESERVE` command page never appeared. The problem was not a lack of documentation; the retriever did not have a direct path to the command as an entity.\n\nV1 represented pages, sections, and command references through one broad entity: `RedisioDoc`. The agent could search it by text or meaning, filter by a broad topic, or fetch a document if it already knew the ID. That worked for many questions, but it treated different retrieval tasks as if they were the same. A question might require:\n\n- An exact command reference.\n- A page overview to find the right topic.\n- A focused passage to support the answer.\n- More evidence from the page where a useful passage was found.\n\nThese tasks need different kinds of information. A short passage is useful evidence, but may lack context. A page summary helps with discovery, but may be too broad to cite. A command has a clear name and structure, so searching for it as general prose can introduce unrelated matches.\n\nThat is what happened with `BF.RESERVE`. The agent knew the name of the command, but V1 could not treat it as a command and go directly to its record. It searched the wider documentation collection instead, where “reserve” also appeared in Kubernetes and REST API pages.\n\n## Three entities connected by retrieval paths\n\nThe `BF.RESERVE` trace suggested a more specific solution, where we stop asking one entity to handle every kind of retrieval. In our next design, we split the documentation into three entities:\n\n- `RedisioChunk` holds focused passages used as answer evidence.\n- `RedisioPage` holds page summaries used for discovery.\n- `RedisioCommand` holds structured command references for precise lookup.\n\nThis gave the agent better choices, but the entities still needed to work together. We also declared an explicit `relationship` link between pages and chunks, and between pages and commands. These relationships used the stable `page_id` shared by the records.\n\n**[Image placeholder]** Entity relationship diagram (RedisioPage ↔ RedisioChunk ↔ RedisioCommand, both relationships join on page_id). Upload from the source draft. Suggested alt text: “RedisioPage for broad discovery links to RedisioChunk for answer evidence and RedisioCommand for exact command lookup, joined on page_id.”\n\nThe added relationships made joins part of the retrieval model and the agent could now follow clearer paths. For example:\n\n- For a direct question, search chunks and return the strongest passages.\n- For a broad question, discover a page from its summary, take its `page_id` , and retrieve chunks from that page.\n- For a command question, fetch the command record directly, then retrieve related prose when more explanation is needed.\n\nThe agent could now choose a route based on the question instead of treating every request as the same kind of document search. The shift was not simply from one index to three indexes. We gave the agent a small map of the documentation: distinct places to search and stable paths for moving between them.\n\n## What the results showed\n\nWe compared V1 and V2 on the same 62-question retrieval set. We used `gpt-5.4-mini` to power the retrieval agent, and both had the same recursion limit of 16 (i.e. the agent can only take a maximum of 16 steps for one question before it hits the safety cap and is stopped).\n\nOur main ranking metric was nDCG@5. Recall only asks whether an expected page appeared somewhere in the results. nDCG also rewards placing it near the top, and it gives more credit to primary sources than to related ones, which matches how our golden dataset is labeled. Rank matters here because the agent reads results in order, and a correct page buried at position nine is much less likely to be used or cited than one at position one. We report recall@5 alongside it to show whether the right evidence appeared at all.\n\nIn simple terms, nDCG measures whether the expected documentation appeared near the top of the first five results. Higher is better. V2 was stronger overall:\n\n| Question group | Questions | V1 nDCG@5 | V2 nDCG@5 | \n|---|---|---|---|\n| Simple lookups | 11 | 0.428 | 0.740 | \n| Multi-step questions | 8 | 0.533 | 0.744 | \n| Hard cases | 25 | 0.484 | 0.680 | \n| Chat-style questions | 18 | 0.375 | 0.405 | \n| **Overall** | **62** | **0.449** | **0.619** | \n\nThe latest V2 run improved retrieval across every question group. The largest gain came from simple lookups, where exact command and entity-specific retrieval paths helped the expected sources reach the top results. Multi-step questions also improved strongly, suggesting that the agent was better able to gather evidence from more than one part of the documentation.\n\nThe improvement also held beyond the first five results. Overall nDCG@10 increased from 0.493 to 0.658, while recall@5 increased from 0.610 to 0.743.\n\nThe richer retrieval surface required more work from the agent:\n\n| Runtime measure | V1 | V2 | \n|---|---|---|\n| Median latency | 6.768s | 10.481s | \n| p95 latency | 10.068s | 17.946s | \n| Median tool calls | 2 | 4 | \n| p95 tool calls | 4 | 8 | \n\nV2 found and ranked better evidence, but it made more tool calls and took longer. That extra latency matters most in the slowest requests. The public `ask` path has a 25-second request limit. At p95, V1 used about 40% of that budget, while V2 used about 72%. That left less room for network delays, provider variance, or another retrieval step. Therefore, the next step is not to undo the richer model, but to make navigation more efficient; for example, by reducing unnecessary tool calls and ending retrieval once enough evidence has been found.\n\n## What we learned so far\n\n### Relationships turn retrieval into navigation\n\nSplitting the corpus into pages, chunks, and commands gave the agent better places to search. Connecting them made each result useful as a next step.\n\nA page search did not need to end with a page summary. Its `page_id` could be passed to the chunk filter to retrieve evidence from that page. A command record could use the same page identity to connect precise command details with supporting prose.\n\nWithout those paths, the agent has to infer the connection, construct the right identifier, and remember the join from its prompt. Explicit relationships move that knowledge into the retrieval model. The solution design does not choose or execute the route for the agent, but it makes the available route clear and dependable.\n\nThis was an important shift in how we thought about retrieval quality. A good result is not only relevant; it should also make the next useful retrieval move easier.\n\n### Model retrieval around real user intent\n\nChunking alone does not create a retrieval strategy. A command question, a broad discovery question, and a request for supporting evidence need different result shapes. Modeling those shapes as separate entities gave the agent a way to choose what it needed before searching, instead of sorting out the meaning of broad results afterward.\n\n### Stable IDs are core infrastructure\n\nRelationships only work when both sides agree on identity. Stable page, section, chunk, command, anchor, and URL IDs allowed results from one tool to become inputs to another. They also made citations, evaluation, and debugging more reliable. We would design this identity system earlier next time.\n\n### Retrieval and answer quality need separate tests\n\nUsers experience the final answer, so answer quality still matters. But evidence-level tests tell us whether a change to the schema, index, or routing actually improved retrieval. Using both views keeps a fluent model from hiding weak search.\n\n### Better retrieval can require more retrieval\n\nV2 improved ranking overall, but it also increased tool use and slow-tail latency. The next step is to use its tools more selectively: route command questions directly, use page discovery only when needed, and set deterministic criteria for the LLM to stop once the evidence is strong enough.\n\n## A better map of the docs\n\nWe began by asking how to retrieve the right Redis document.\n\nWe ended up with a better question: what map of the documentation helps an agent take the right next step?\n\nSometimes that step is finding a passage. Sometimes it is discovering a page. Sometimes it is looking up a command. Sometimes it is following a link from a broad result to evidence ready for citation.\n\nContext Retriever was the mechanism we used to model the entities and relationships, but the broader lesson is not to add more entities or tools by default. It is to model the few retrieval moves that match the real structure of the corpus, connect them with stable identifiers, and measure whether the agent actually uses them well.\n\nThe move from one document type to a connected documentation surface did not remove complexity. It made that complexity easier to see, measure, and improve.\n\nFor agentic retrieval, that may be the most useful result of all.\n\n## Get started with Redis today\n\nSpeak to a Redis expert and learn more about enterprise-grade Redis today.", "url": "https://wpnews.pro/news/connecting-coding-agents-to-the-right-documentation-with-a-navigable-context", "canonical_source": "https://redis.io/blog/navigable-context-layer-for-coding-agents/", "published_at": "2026-10-29 00:00:00+00:00", "updated_at": "2026-10-31 00:49:20.140152+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-search", "developer-tools"], "entities": ["Redis", "redis.io/docs", "Context Retriever", "RedisioDoc", "RetrievalTrace"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/connecting-coding-agents-to-the-right-documentation-with-a-navigable-context", "markdown": "https://wpnews.pro/news/connecting-coding-agents-to-the-right-documentation-with-a-navigable-context.md", "text": "https://wpnews.pro/news/connecting-coding-agents-to-the-right-documentation-with-a-navigable-context.txt", "jsonld": "https://wpnews.pro/news/connecting-coding-agents-to-the-right-documentation-with-a-navigable-context.jsonld"}}