{"slug": "enterprise-ai-search-can-t-answer-the-only-question-that-matters-why", "title": "Enterprise AI search can't answer the only question that matters: \"why?\"", "summary": "A developer building enterprise AI search argues the category is optimizing for the wrong question, since engineering queries like \"why is checkout rate-limited to 20 requests per second?\" require joining facts across ADRs, Slack threads, and source code rather than ranking documents. The developer describes building a claim checker that re-reads every cited source and grades each claim supported, partial, unsupported, or unverifiable, with a free AST-based structural pass before the model-based one, and warns that agents consuming company knowledge over MCP will silently act on wrong answers if verification is absent.", "body_md": "Every company search tool demos the same way.\n\nSomeone types *\"what's our refund policy?\"*, the AI returns a tidy paragraph with three source chips, the room nods, and procurement starts a twelve-week evaluation.\n\nNow watch an actual engineer's Tuesday:\n\n*\"Why is checkout rate-limited to 20 requests per second?\"*\n\nThat answer is not in a document. It's in an ADR nobody linked, a Slack thread from November that scrolled away, and one line in `src/rate_limiter.py`. The three facts only mean something **together**. Enterprise search finds the file. It does not find the reason.\n\nThis is the gap I want to argue about, because I think the whole category is optimising for the wrong question — and I say that as someone building in it.\n\nTools like Glean are genuinely good at what they were built for: company-wide search over hundreds of SaaS connectors, with SSO, admin controls and a sales-led contract. If you need one search box over Drive, Jira, Zendesk and Confluence, that is a real and hard problem, and they solved it.\n\nBut engineering questions have a shape those tools don't fit:\n\n| What a search tool optimises | What an engineer actually asks | \n|---|---|\n| \"Find the document about X\" | \"Why is it built this way?\" | \n| \"Summarise this page\" | \"What breaks if I change this?\" | \n| \"Who wrote this doc?\" | \"Who would actually know?\" | \n| Documents ranked by relevance | Facts joined across code, chat, and tickets | \n\nThe second column is not a retrieval problem. It's a **graph** problem plus a **provenance** problem. Ranking documents better does not get you there, no matter how good the embeddings are.\n\nEvery one of these tools shows sources. Almost none of them check that the cited source actually supports the sentence next to it.\n\nThis is not a hypothetical. It's the single most common failure we hit: the model retrieves the right file and then paraphrases something the file does not say. The citation is \"valid\" — the document is real, the retrieval worked — and the answer is still wrong. Worse, the chip next to it makes the wrong answer *more* persuasive.\n\nWe ended up building a checker that re-reads every cited source and grades each claim `supported / partial / unsupported / unverifiable`, with a structural (free, AST-based) pass before the model-based one. The part that mattered wasn't the model. It was the rule that **the checker may never say \"supported\" just because it couldn't run**. If the oracle has no data or the key is missing, the claim stays `unverifiable` and the UI says what was *not* checked.\n\n\"Has a citation\" and \"is true\" are different bars. Most tools ship the first and imply the second.\n\nAsk any of these systems something genuinely undocumented and you get a fluent paragraph assembled from adjacent noise. Almost none of them say *\"nobody has written this down.\"*\n\nThat's the honest answer surprisingly often, and it's actionable in a way a hallucination never is — because the next step is obvious: **go ask the person who knows.** A system that knows who touched that code and who ingested that repo can name them, ask them in Slack, and file the reply back into the wiki so the next person doesn't have to ask. Not knowing isn't the failure. Pretending not to notice is.\n\nMost of these products are a search box on a web page. But in 2026 the heaviest consumer of company knowledge isn't a person typing a query — it's the coding agent in your editor, which will otherwise guess.\n\nIf your company brain isn't reachable over MCP from Claude Code or Cursor, your agents are working from vibes while your humans get the good data. That's backwards: the agent is the one that will *silently* act on a wrong answer.\n\nThe pricing model is the other tell. Per-seat enterprise contracts with inference bundled in mean:\n\nBring-your-own-key flips that. If the user pays their own provider directly, the tool has no reason to pad answers, and a verification pass that says \"this claim is unsupported\" becomes a feature rather than an admission.\n\nI'd be a hypocrite to write about honest answers and then strawman the competition, so:\n\nIf all you need is one search box over every SaaS tool your company owns, this post isn't for you. If your hardest questions start with \"why is this built this way\", it is.\n\nBefore you sign anything, ask the tool one question you already know the answer to — something that required a decision, not just a lookup. `Why is X configured this way?`\n\nThen grade it:\n\nMost tools pass 1 and fail 3 and 4. That's the whole thesis of this post.\n\nWe're building [Enhanciar](https://enhanciar.in?utm_source=devto&utm_medium=article&utm_campaign=vswhy) around those five checks — a company brain for engineering teams, with every answer cited to file:line, every citation re-verified, and the whole thing reachable over MCP. It's waitlist-only right now.\n\nI'm genuinely interested in disagreement on point 1: **is an unverified citation better than no citation at all?** I've argued myself into \"no — it's worse, because it buys unearned trust.\" Tell me why I'm wrong.", "url": "https://wpnews.pro/news/enterprise-ai-search-can-t-answer-the-only-question-that-matters-why", "canonical_source": "https://dev.to/enhanciar_ai_96a4ba4877e3/enterprise-ai-search-cant-answer-the-only-question-that-matters-why-k0e", "published_at": "2026-09-22 16:53:17+00:00", "updated_at": "2026-09-22 17:23:01.300434+00:00", "lang": "en", "topics": ["ai-search", "ai-agents", "agent-protocols", "ai-tools", "ai-infrastructure"], "entities": ["Glean", "Claude Code", "Cursor", "MCP", "Slack", "Confluence", "Jira", "Zendesk"], "alternates": {"html": "https://wpnews.pro/news/enterprise-ai-search-can-t-answer-the-only-question-that-matters-why", "markdown": "https://wpnews.pro/news/enterprise-ai-search-can-t-answer-the-only-question-that-matters-why.md", "text": "https://wpnews.pro/news/enterprise-ai-search-can-t-answer-the-only-question-that-matters-why.txt", "jsonld": "https://wpnews.pro/news/enterprise-ai-search-can-t-answer-the-only-question-that-matters-why.jsonld"}}