{"slug": "freshgauge-a-freshness-control-plane-inside-the-llm-gateway", "title": "FreshGauge: a freshness control plane inside the LLM gateway", "summary": "A developer built FreshGauge, a plugin for the Bifrost LLM gateway that extracts factual claims from model completions, verifies them against live web evidence via Tavily, and scores them by authority-weighted recency. The plugin runs in three modes — shadow (post-answer logging), correct (appending dated corrections inside the PostLLMHook at zero Tavily credits by reading a SQLite ledger), and brief (pre-empting stale answers in HTTPTransportPreHook with one search) — exploiting the asymmetry that detection needs the web but enforcement only needs memory of prior detections. Admin routes inherit Bifrost's virtual-key auth because HTTPTransportPreHook runs after gateway authentication.", "body_md": "Every LLM answer is a snapshot of the model's training data. Ask for a price, a current model name, a CEO, or a fine cap and you get something that *was* true — stated with exactly the same confidence as something that still is. Retrieval doesn't fix this on its own: RAG grounds an answer in documents, but nobody checks whether the document the answer leans on is two years old.\n\nFreshGauge is a Bifrost gateway plugin that does that check on the way out. Every completion that passes through the gateway gets its factual claims extracted, verified against live web evidence, scored by *authority-weighted recency*, and then acted on — logged, corrected in flight, or pre-empted with a dated brief before the model is even called.\n\nThis is a write-up of how it's built, the decisions that mattered, and the traps that cost real debugging time.\n\nYou could build this as a wrapper around one SDK. It breaks the moment a team uses three providers, streams tokens, or fails over between models. A gateway sees all of that by construction: one process, one config, every request.\n\nBifrost's plugin ABI gives exactly the hooks needed:\n\n`PostLLMHook` — the response, after the provider call. Where the audit is born.`HTTPTransportPreHook` — every HTTP request to the gateway. Where the audit API lives.\nBecause `HTTPTransportPreHook` runs *after* Bifrost's auth, the plugin's admin routes inherit virtual-key auth for free. There is no second auth system to get wrong.\n\n```\n                         ┌──────────────────────────────────────────────┐\n  client ──POST─────────▶│            Bifrost gateway (:8080)            │\n  /v1/chat/completions   │                                              │\n                         │  HTTPTransportPreHook ──▶ /audit, /stats, …  │\n                         │        │                       (auth'd by VK) │\n                         │        ▼                                     │\n                         │  provider call (OpenAI / Anthropic / X / …)  │\n                         │        │                                     │\n                         │  PostLLMHook                                 │\n                         └────────┬─────────────────────────────────────┘\n                                  │ 1. extract claims (≤8)\n                                  │ 2. correct in flight?  ◀── ledger read (0 credits)\n                                  │ 3. dedupe by request/content\n                                  ▼\n                        ┌──────────────────────┐\n                        │  auditor (2 workers) │\n                        │  per claim:          │\n                        │   Tavily search  ────┼──▶ api.tavily.com (1–2 credits)\n                        │   fetch + quote      │\n                        │   stance: +1 / −1 / 0│\n                        │   vote = auth·e^(−age/τ)\n                        └──────────┬───────────┘\n                                   ▼\n                        ┌──────────────────────┐\n                        │ SQLite ledger        │\n                        │ audits · claims ·    │\n                        │ evidence · metrics   │\n                        └──────────┬───────────┘\n                                   ▼\n                   GET /audit/{id}.html  (flat paper report)\n                   GET /stats            (corrections, grounded)\n```\n\nThree modes sit on one engine:\n\n| Mode | When | Cost | What the user sees | \n|---|---|---|---|\n| `shadow` | after the answer | Tavily credits | nothing; the ledger only | \n| `correct` | inside `PostLLMHook` | **0 credits** (ledger read) | dated correction block appended | \n| `brief` | in `HTTPTransportPreHook` | 1 Tavily search | answer born current | \n\nThe cost asymmetry is the point. Detection needs the web. Enforcement does not — it needs the *memory* of what was already detected.\n\nThe extractor pulls sentences that make checkable assertions about the world and classifies them:\n\n```\nconst (\n    Pricing     Category = \"pricing\"\n    APIVersion  Category = \"api_version\"\n    Leadership  Category = \"leadership\"\n    Regulation  Category = \"regulation\"\n    Limits      Category = \"limits\"\n)\n```\n\nCategories are not cosmetic. Each one carries a confidence half-life, τ, that decides how fast evidence decays:\n\n```\nfunc Tau(c extract.Category) float64 {\n    switch c {\n    case extract.Pricing:\n        // A five-week-old vendor price page is still current. At tau=30 a 39-day-old\n        // page scores 0.27 and fails MinVote, which reports a plainly stale price as\n        // UNVERIFIABLE — abstention where the evidence is good enough to decide is a\n        // worse error than a slightly generous half-life.\n        return 120\n    case extract.Limits:\n        return 180\n    case extract.APIVersion:\n        return 90\n    case extract.Leadership:\n        return 365\n    case extract.Regulation:\n        return 730\n    }\n    return 180\n}\n```\n\nA statute moving on a decade timescale and a price sheet moving on a quarterly one cannot share a decay constant. That table is the whole domain model, and it is the difference between a detector that argues and one that abstains.\n\nMost \"is this stale?\" implementations count sources. Counting is wrong: ten 2023 blog posts should not outvote one vendor pricing page published last month.\n\nSo each piece of evidence casts a weighted vote:\n\n```\n// Recency decays from 1.0 at the publish/fetch date to ~0.37 one tau later.\nfunc Recency(e Evidence, cat extract.Category, now time.Time) float64 {\n    date := e.Published\n    if date.IsZero() {\n        date = e.Fetched\n    }\n    if date.IsZero() || date.After(now) {\n        return 1\n    }\n    ageDays := now.Sub(date).Hours() / 24\n    return math.Exp(-ageDays / Tau(cat))\n}\n\n// Vote is the authority-weighted recency used for every decision.\nfunc Vote(e Evidence, cat extract.Category, now time.Time) float64 {\n    if e.Stance == 0 {\n        return 0\n    }\n    return clamp01(e.Authority) * Recency(e, cat, now)\n}\n```\n\nDecisions follow from that single scale:\n\n`bestCon >= MinVote (0.35)` `bestCon >= bestSup` → CONTRADICTED`bestSup >= MinVote` → SUPPORTED\nAnd the freshness score is not a vibe — it's recomputable by hand from the ledger:\n\n```\nf := 100 * (1 - math.Min(1, bestCon))\n```\n\nFor the recorded demo: the vendor page (`acme.com/pricing`, authority 1.0, published 2026-10-02) says $35 and contradicts; a 2024 Medium post (authority 0.4) supports the old $20. The contradiction's vote wins by an order of magnitude, and the score is 28/100 — recent enough to be *decisive*, old enough that the claim it contradicts is plainly stale.\n\nThe first live run against `anthropic.com` flagged **`$2 / million input tokens` as CONTRADICTED**. The page did say `$15 / million input tokens` — for *a different model*, in a multi-model table.\n\nNaive comparison of \"a number appeared near the claim\" turns every pricing table into a contradiction factory. The fix gates comparison on three cheap invariants before any number is compared:\n\n`subjectOverlap`, `sameSubject`).` Claude Sonnet 5` ≠ `Claude Sonnet 4`.` per million input tokens` ≠ `per million output tokens` ≠ `per month` (`priceKind`, `taggedPrices`).\nAnd the scan is line-first: candidate evidence is the single line containing the number, not the whole page. After the fix, the same page and the same single Tavily search produce `$15/M input` → CONTRADICTED and `$2/M input` → SUPPORTED. There's a regression test for exactly that pair.\n\n`response_mode: \"correct\"` rewrites the answer before the caller sees it. The correction is drawn entirely from the ledger — the plugin's own memory of contradictions it has already established — so it costs **zero** Tavily credits.\n\n```\nclaims := extract.Claims(text, p.cfg.MaxClaims)\n\n// In-flight correction runs before the audit is enqueued, and only ever reads the local\n// ledger. Order matters: the audit below must never see its own correction block as a\n// new claim, or every corrected answer would spawn fresh work for text the plugin wrote\n// itself.\nif !streaming && p.cfg.ResponseMode == \"correct\" {\n    p.correctInFlight(resp.ChatResponse, claims)\n}\n```\n\nTwo details earn their comments:\n\n`stripCorrection`). Otherwise the block the plugin appended becomes a claim on the next audit and the plugin audits itself forever.`normalizeClaim` collapses punctuation and casing so the same fact expressed twice hits the same row; priors are capped by `correction_max_age_hours` (24h default) so a correction never cites evidence the operator considers expired.\nThe output is appended under a fixed marker, three lines max by default:\n\n```\n---\n**Freshness check (FreshGauge).**\n- Correction: \"Acme Pro costs $20 per month\" is stale. https://acme.com/pricing says\n  \"Pro — $35 per user / month\" (authority 1.00, published 2026-10-02).\n```\n\n`/stats` exposes `corrections` and `grounded_requests` next to audit totals, so \"the gateway changed answers\" is a measurable event rather than a claim in a README.\n\n`grounding_mode: \"brief\"` runs in `HTTPTransportPreHook`, only for freshness-sensitive questions (` looksFreshnessSensitive`), inside a time box (` grounding_timeout_ms`), and injects a short block of *dated* evidence into the request:\n\n```\nCurrent evidence (verified 2026-11-21):\n- acme.com/pricing (2026-10-02): \"Pro — $35 per user / month\"\n```\n\nThe model answers from that instead of from its weights. One search, opt-in, and it shows up on `/stats` as `grounded_requests`.\n\nStream chunks carry a `Delta`, not a `Message`. Reaching for `Choices[0].Message` dereferences a nil embedded pointer — inside a hook, that surfaces as a recovered panic in the host log and a silently missing audit, which is the worst failure mode: it looks like it works.\n\nThe hook detects the stream case and assembles deltas per request:\n\n```\nswitch {\ncase c0.ChatNonStreamResponseChoice != nil:\n    text = stripCorrection(messageText(resp.ChatResponse))\ncase c0.ChatStreamResponseChoice != nil:\n    streaming = true\n    text = p.streamText(ctx, c0)   // buffers deltas until finish_reason\n}\n```\n\nStreams are audited once, at finish, and never mutated — there is no message object to amend, so `correct` is non-streaming by design. Buffers are reaped on completion, and there's a test that feeds a synthetic SSE sequence through the hook to keep it that way.\n\nDeduplication also needs care: the host does not always put a request ID on the context, so the fallback is `content-<hash>-<minute>`. Two failover attempts producing identical prose collapse into one audit; a genuine repeat a day later gets a fresh one with fresh evidence dates.\n\nBifrost plugins are Go `.so` files loaded into the host process. That imposes an exact-match ABI: same Go toolchain **and** an identical module graph, `replace` directives included. A one-character drift in a transitive dependency and the host refuses to load the plugin — with an error that does not name the offending package.\n\nWhat worked:\n\n`go1.27.3`) and Bifrost (` core v1.15.2`, `transports v2.5.1`).` replace` for `mcp-go` exactly:\n`replace github.com/mark3labs/mcp-go => github.com/maximhq/mcp-go v0.0.0-20261102181017-b7da3bc145a1`\n`.so` loads before touching logic.\nRuntime config lives in `<app-dir>/config.json`; the plugin reads it in `Init(raw any)` and never breaks a completion when its own storage fails:\n\n```\nauditID, cerr := p.st.CreateAudit(model, requestID, claims)\nif cerr != nil {\n    return resp, bifrostErr, nil // never break a completion because the ledger failed\n}\n```\n\nA freshness check that takes down the gateway is worse than no freshness check.\n\nMounted through `HTTPTransportPreHook`, so it inherits gateway auth:\n\n```\nGET  /health\nGET  /stats                 # audits, claims, contradictions, corrections, grounded\nGET  /audits                # recent list\nGET  /audit/{id}            # JSON\nGET  /audit/{id}.html       # flat paper report, printable\nPOST /audit                 # { \"text\": \"…\" } → { \"audit_id\": \"…\", \"status\": \"pending\" }\nGET  /expiring              # corrections about to age out of the ledger window\ncurl -s localhost:8091/audit -d '{\"text\":\"Acme Pro costs $20 per month.\"}' | jq -r .audit_id\n# → a_58d9ce122048 ; pending is async, poll /audit/{id}\n```\n\nThe HTML report is deliberately unglamorous: paper background, hairline rules, one block per claim with the decisive evidence quote, its authority, its publish date, and the arithmetic that produced the score.\n\nA detector is only as good as its labeled set, and the temptation is to label after seeing predictions. The bench avoids that:\n\n`bench/claims.csv` — one claim per row with `is_true_now`, `label_url`, `label_quote`, `label_date`, `adversarial`.` bench/run.py` posts each claim to `POST /audit` against live Tavily; `bench/score.py` compares predicted to label and reports accuracy plus per-verdict precision/recall.\nThe interesting failure mode is not \"wrong verdict on a stale claim\" — it's \"confident verdict on a claim whose subject the search never actually resolved.\" Abstention is a feature, and the report says `UNVERIFIABLE` rather than inventing certainty.\n\n`authority × e^(−age/τ)` per category, with an explicit abstain band.\nThe whole thing is a few thousand lines of Go: extractor, scorer, Tavily client, SQLite ledger, auditor workers, an HTML report, and two hooks.\n\nCode & more: [https://www.dailybuild.xyz/project/321-fresh-guage](https://www.dailybuild.xyz/project/321-fresh-guage)", "url": "https://wpnews.pro/news/freshgauge-a-freshness-control-plane-inside-the-llm-gateway", "canonical_source": "https://dev.to/harishkotra/freshgauge-a-freshness-control-plane-inside-the-llm-gateway-533e", "published_at": "2026-11-22 04:09:25+00:00", "updated_at": "2026-11-22 04:12:11.972540+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "ai-tools", "developer-tools"], "entities": ["FreshGauge", "Bifrost", "Tavily", "SQLite", "OpenAI", "Anthropic"], "also_reported_by": [], "takeaway": "Enforce freshness corrections from a cached ledger read (0 Tavily credits) inside PostLLMHook rather than re-searching the web, reserving paid searches for detection and pre-emptive briefs.", "recommendation": {"advice": "Verify factual claims against live evidence in shadow mode first, then enforce corrections from the stored ledger to avoid paying search credits on every response.", "playbook": "output-verification", "playbook_url": "https://wpnews.pro/playbooks/output-verification"}, "alternates": {"html": "https://wpnews.pro/news/freshgauge-a-freshness-control-plane-inside-the-llm-gateway", "markdown": "https://wpnews.pro/news/freshgauge-a-freshness-control-plane-inside-the-llm-gateway.md", "text": "https://wpnews.pro/news/freshgauge-a-freshness-control-plane-inside-the-llm-gateway.txt", "jsonld": "https://wpnews.pro/news/freshgauge-a-freshness-control-plane-inside-the-llm-gateway.jsonld"}}