{"slug": "a-cited-ai-answer-can-still-be-wrong-the-retrieval-bug-we-found-in-zettavector", "title": "A cited AI answer can still be wrong: the retrieval bug we found in ZettaVector", "summary": "ZettaVector, an AI search and retrieval company, disclosed a bug in its contact recovery feature that returned a wrong email address with a valid citation, because the system selected the first syntactically valid email without checking its context. The company fixed the issue by replacing selection with a ranking system that scores email candidates based on nearby text signals, such as contact labels and footer indicators, and rejects candidates that appear to be demo data. The bug highlights that citations prove origin, not meaning, and that grounded answers can still be contextually incorrect.", "body_md": "We asked our agent a simple question: “How can I get in touch?” It returned `email@joes.com`\n\n. The answer included a valid citation to the company homepage.\n\nIt was also wrong.\n\nThe address belonged to a fictional lead in a product demo. The real business address appeared in the footer of the same page. Our system found both addresses, then assigned the wrong role to the first one.\n\n## The answer passed the obvious checks\n\nThis was not a fabricated address. It was not a failed crawl. It was not a missing chunk. The retrieved page contained the returned text, so the citation validator accepted it.\n\nSeveral parts of the grounding pipeline were working:\n\n- The homepage had been crawled and indexed.\n- The retrieval result contained relevant contact evidence.\n- The returned email existed in the cited source.\n- The source reference matched the retrieved chunk.\n\nThe failure happened after retrieval. The contact recovery path scanned the evidence in citation order and selected the first syntactically valid email.\n\nSIMPLIFIED RETRIEVED EVIDENCE\n\n[1] Live results dashboard Joe's Pizza · 87 · email@joes.com Bright Smile Dental · 82 · contact@smile.com [2] NexalLabs · Enterprise · Feedback wchisasa@outlook.com © 2026 NexalLabs. All rights reserved.\n\nThe first chunk described product output. The second chunk described the business. A regular expression could identify each email, but it could not identify what each email represented.\n\n## The flawed assumption\n\nThe original recovery behavior was equivalent to this simplified code:\n\n``` js\nfor (const source of retrievedEvidence) {\n  const email = findFirstEmail(source.text)\n\n  if (email) {\n    return email\n  }\n}\n```\n\nThis logic answered one question:\n\nDoes the retrieved evidence contain a valid email address?\n\nThat check was necessary, but it was too weak. Contact recovery also needed to ask whether the page presented that address as an official way to reach the business.\n\n## A citation proves origin, not meaning\n\nCitations are useful because they let a reader inspect the source behind an answer. They do not prove that the system interpreted the source correctly.\n\n| CHECK | WHAT IT ESTABLISHES |\n|---|---|\n| Valid citation | The cited source contains the referenced text. |\n| Grounded claim | Retrieved evidence supports the words in the answer. |\n| Contextually correct answer | The evidence has the role and meaning assigned to it. |\n\nOur answer satisfied the first two checks. It failed the third. The source contained `email@joes.com`\n\n, but the page presented it as sample lead data, not the company contact address.\n\n## We changed selection into ranking\n\nThe fix evaluates every retrieved email candidate. Each candidate receives a score based on nearby text. The system then sorts by score and uses source order only to break a tie.\n\nStrong business-contact signals add weight:\n\n- Direct labels such as “contact,” “email,” “support,” or “sales”\n- Contact phrases such as “get in touch”\n- Business context such as “enterprise” or “feedback”\n- Footer signals such as a copyright mark or “all rights reserved”\n\nDemo and interface signals subtract weight:\n\n- “Demo,” “sample,” “prospect,” or “lead”\n- “Dashboard,” “results table,” or “outreach hook”\n- Table headings such as “business,” “score,” or “website quality”\n- Several email addresses inside the same chunk\n\nThe production logic also applies a minimum score to email candidates. If every address looks like example data, contact recovery returns nothing. The agent does not expose the least bad candidate.\n\n``` js\nconst candidates = evidence\n  .flatMap(findEmailCandidates)\n  .filter(candidate => candidate.score >= minimumScore)\n  .sort(byScoreThenSourceOrder)\n\nreturn candidates[0] ?? null\n```\n\nThis is simplified code. The important change is the decision boundary. Finding a matching string is no longer enough.\n\nRESULT AFTER CONTEXT SCORING\n\nDemo table candidate email@joes.com score: rejected Business footer candidate wchisasa@outlook.com score: selected\n\n## The refusal case matters as much as the success case\n\nWe added two regression tests around the production failure.\n\n- Place multiple realistic demo addresses before the footer address. The footer address must win.\n- Provide a page that contains only a demo lead address. The system must say that the available information does not specify a business email.\n\nThe second case protects the more important behavior. When evidence is ambiguous or low quality, the system should not turn a plausible string into an official claim.\n\n## What we took from the bug\n\n### Retrieval order is not authority order\n\nSimilarity ranking finds text related to the question. It does not establish which part of a page is authoritative. A product demo can be highly similar to a contact query because it contains names, companies, and email addresses.\n\n### Example content is dangerous evidence\n\nRealistic examples are useful to visitors. They are also easy for an automated system to mistake for business facts. Demo tables, testimonials, templates, and screenshots need stronger contextual treatment.\n\n### Deterministic code still needs semantic safeguards\n\nThis failure came from a deterministic recovery path, not a model response. Removing model variability did not remove the need to reason about meaning.\n\n### Grounding is a pipeline\n\nA grounded system needs more than retrieval and citations. It also needs exact source binding, claim validation, context-aware selection, and a safe refusal path. A pass at one stage does not guarantee a correct final answer.\n\n## What this fix does not solve\n\nThis change addresses contact selection when retrieved pages contain several email candidates. It does not solve every retrieval or grounding failure.\n\nAmbiguous phone-number questions, sentence fragments, and repeatability across identical queries require separate tests. We track those cases independently. The goal is not to label the pipeline “solved.” The goal is to make each failure observable, reproducible, and harder to repeat.\n\n## The standard we want\n\nZettaVector answers questions on behalf of a real business. A response can affect whether a visitor trusts that business or reaches the right person. “The text appeared somewhere on the page” is not a sufficient standard.\n\nA useful cited answer needs the right source, the right claim, and the right interpretation. This bug gave us the source and the claim. Fixing it required us to handle the meaning too.", "url": "https://wpnews.pro/news/a-cited-ai-answer-can-still-be-wrong-the-retrieval-bug-we-found-in-zettavector", "canonical_source": "https://zettavector.com/blog/cited-ai-answer-can-still-be-wrong", "published_at": "2026-08-27 16:50:12+00:00", "updated_at": "2026-08-27 17:19:27.891036+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-products", "ai-tools"], "entities": ["ZettaVector"], "alternates": {"html": "https://wpnews.pro/news/a-cited-ai-answer-can-still-be-wrong-the-retrieval-bug-we-found-in-zettavector", "markdown": "https://wpnews.pro/news/a-cited-ai-answer-can-still-be-wrong-the-retrieval-bug-we-found-in-zettavector.md", "text": "https://wpnews.pro/news/a-cited-ai-answer-can-still-be-wrong-the-retrieval-bug-we-found-in-zettavector.txt", "jsonld": "https://wpnews.pro/news/a-cited-ai-answer-can-still-be-wrong-the-retrieval-bug-we-found-in-zettavector.jsonld"}}