Why Vector Search Breaks Production: Building a 2-Hop Relational Context Engine in Sanity

A developer built SHIPCHECK, a deterministic 2-hop relational context engine on the Sanity Content Lake, to replace vector similarity search for engineering dependency queries. Benchmarked against a simulated Pinecone/LangChain RAG pipeline over the same 18 documents, the vector approach returned a false "SAFE TO SHIP" verdict and missed a Payment SDK version requirement, while the GROQ-based graph traversal detected the cluster collision and returned a hard blocker. The engine also resolves documentation drift by authority level and supports in-memory graph mutation for simulating fixes.

This is a submission for the Sanity Challenge, Path Two: Build a Knowledge Base or Context Engine https://dev.to/challenges/sanity-2026-09-16 . The AI community has spent the last three years building Retrieval-Augmented Generation RAG on top of vector similarity search Pinecone, Chroma, pgvector . For prose and chat applications, cosine similarity on text chunks works well. For engineering infrastructure, vector search is a recipe for outages. Vector databases compress text into high-dimensional vectors. In doing so, they lose: Service A v4 strictly requires Service B = v3.0 . For Path Two , we built a deterministic 2-hop relational context engine on top of the Sanity Content Lake . Instead of flattening our documentation into vectors, we model organizational knowledge as a structured, queryable graph of microservices, version constraints, policies, and semantic relationships. When an engineer or autonomous coding assistant asks: "Can I upgrade payment service from v2 to v4 tonight?" Our engine doesn't guess with embeddings. It executes deterministic GROQ graph traversals that surface the hidden dependency chain, detect cluster pod collisions, and resolve documentation contradictions with 100% mathematical precision. We designed 18 interconnected documents in Sanity 70rd1u6b across three core schema types: component versionConstraint targetComponent , requiredComponent , operator , version , criticality . knowledgeEntry contradicts : Directed references pointing to documents this entry intentionally overrides. isExceptionOf : Encodes conditional policy exemptions e.g., emergency security hotfixes . authorityLevel : An integer 1–10 providing deterministic arbitration when documentation drifts. Here is the core GROQ query executed by our context engine to resolve 2-hop dependency chains in a single database round-trip: type == "component" && name == $componentName 0 { name, currentVersion, "hop1 constraints": type == "versionConstraint" && targetComponent. ref == ^. id { "requiredServiceName": requiredComponent- name, "requiredClusterVersion": requiredComponent- currentVersion, operator, version, criticality }, "drift audit": type == "knowledgeEntry" && references ^. id | order authorityLevel desc { title, authorityLevel, sourceType, "contradicts": contradicts - title, "isExceptionOf": isExceptionOf- title, body } } This single query traverses: Payment Service . requires Payment SDK = v3.0.0 . Payment SDK v2.4.1 . We benchmarked our Sanity Context Engine against a standard vector similarity search simulating Pinecone / LangChain RAG over the exact same 18 documents. User Query: "Can I upgrade payment service from v2 to v4 tonight?" | Evaluation Metric | Naive Vector Search Pinecone/RAG | SHIPCHECK Sanity Context Engine | |---|---|---| | Verdict | ❌ SAFE TO SHIP False Positive | ✅ DO NOT SHIP YET Hard Blocker | | Why it reached this result | Matched keywords "payment service" and "upgrade". Found documents saying v4 has great new features. Completely missed the SDK requirement because the word "SDK" was not in the prompt. | Traversed Hop-1 to find Payment SDK = v3.0.0 . Traversed Hop-2 to find cluster running v2.4.1 . Detected collision immediately. | | Documentation Drift | Hallucinated a merge of XML and JSON specs. | Resolved by authority: Release Notes Auth 10 explicitly supersedes Stale Wiki Auth 4 . | | Production Outcome | P0 Outage: Token HMAC verification failures across all checkout pods. | 0 Outages: Outage prevented before code merged. | Our engine supports in-memory graph mutation . When an engineer clicks "Simulate Fix" , the engine re-evaluates the GROQ traversal with Payment SDK v3.1.0 applied to the cluster representation. The 2-hop collision clears, all constraints evaluate to green, and the verdict live-flips to SAFE TO SHIP . lib/engine/groqTraversal.ts sanity/schemaTypes/ lib/engine/naiveSearchStub.ts When building this context engine for AI coding assistants via MCP JSON-RPC , we noticed: 70rd1u6b production 2024-01-01 Built with ❤️ for the Sanity Context MCP Hackathon Path Two .