# Nobody Alerts on Silence: Wiring Sentry Into an LLM Pipeline

> Source: <https://dev.to/mrviduus/nobody-alerts-on-silence-wiring-sentry-into-an-llm-pipeline-12lo>
> Published: 2026-08-11 12:00:00+00:00

*This is a submission for DEV's Summer Bug Smash: Clear the Lineup powered by Sentry.*

🔨 #bugsmash, week by week:

[a 390% CPU hour nobody noticed],[a state machine with no exit],[a backup that leaked 156 GB]. Week four is the finale: I wired monitoring into the pipeline that produced all three — and its best catch was itself.

TextStack is an open-source reader for technical books, built in .NET: an ASP.NET Core API, a background Worker, PostgreSQL + pgvector, React on top. The LLM pipeline does translation, word explanations, "Ask this book" RAG, and three production agents (Enrichment, Librarian, Tutor), routed between a local Ollama and OpenAI by a config-driven router. The code is public: [github.com/mrviduus/textstack](https://github.com/mrviduus/textstack).

Three weeks ago a user's PDF fell through my LLM router onto a CPU-only Ollama container instead of GPT-4.1, and my CPU sat at 390% for an hour. Zero exceptions. Zero error logs. Zero alerts. And when I went to see what my existing observability had recorded, the answer was *nothing at all*: the OTLP exporter pointed at an Aspire dashboard container that is profile-gated and doesn't run in production. Every span my services had ever produced in prod had been fired into a closed socket.

**Observability you never read is indistinguishable from observability you never installed.**

The one-line config fix was submission #1. This submission is the fix for the *class* of bug — a system that has no way to make a sound when it does the wrong thing successfully:

Four PRs, all merged to main; 1,363 unit tests, full CI green:

`23505`

on reading-progress upserts, real users losing their place in books**The router now says why.** Route resolution was a `??`

chain that produced a string — *identical* whether an operator deliberately routed a task or it fell off the end onto the default. That chain doesn't just fail to record intent; it destroys it. So it returns two things now:

``` js
private RouteDecision ResolveRoute(string? featureTag)
{
    var matched = RegistryKey(featureTag) ?? ConfigRouteKey(featureTag);
    return matched is not null
        ? new RouteDecision(matched, RouteReason.RouteMatched)
        : new RouteDecision(config["Ai:DefaultProvider"] ?? "openai",
                            RouteReason.DefaultFallback);
}
```

Every LLM call tags its span with `ai.task`

, `ai.provider.resolved`

, and `ai.provider.reason`

= `route_matched | default_fallback`

. "Which model answered this, and did anyone choose it on purpose?" is now a trace query instead of a CPU graph.

**Alert arithmetic.** `pdf.parse`

resolves a route once per *page* with parallelism six — my first version would have turned the original incident into 106 identical Sentry events. Every alarm goes through a throttle keyed on `(task, provider, reason)`

: first hit fires immediately, then one event per hour per distinct problem. The unit test literally counts to 106 and asserts one claim.

**No silent fallback, ever — in either direction.** When the breaker finds Ollama dead, tasks are skipped and stay queued; nothing auto-switches to a paid provider, because that converts an outage into unbounded spend. Provider choice stays 100% config-driven.

**And the first live run found a hole in my own fix.** The startup probe opens the circuit on a one-minute backoff; the backfill worker wakes after a two-minute start delay — by then the circuit is legitimately half-open, and my single up-front gate waved the whole batch through. A per-book re-check turned 38 calls into one:

```
Metadata backfill: enriching 38 user books
Metadata backfill: aborting after 0 enriched / 1 failed — provider 'ollama'
  is unavailable; the remaining candidates stay queued
```

Tests check what you imagined; a live run checks what's there.

**Error Monitoring — what the first 24 hours in production caught:**

`HTTP 429 (insufficient_quota)`

on `/translate`

and `/explain`

— the entire paid surface had been failing for readers for twelve hours. No version of my logs would have surfaced that before a user complained.`PUT /me/progress`

threw `23505: duplicate key value violates unique constraint`

ten times in four hours: a textbook read-then-insert race (session heartbeat + `sendBeacon`

on unload + second device), milliseconds wide, invisible in tests. Fixed in **Custom tags** (`ai.task`

, `ai.provider`

, `ai.failure`

, `agent.name`

, `agent.outcome`

) go through an **allowlist scrubber** — every tag not explicitly blessed dies at the edge, so a future `SetTag("prompt", userText)`

can never leak. A Sentry issue answers "which feature, on which model, is broken?" without opening a trace.

**Tracing** covers agent runs and RAG indexing at 100% sampling (they're the reason I installed this), HTTP at 20%, health checks at 0%. I rejected the deprecated OTel bridge *specifically because* spans leaving through the OpenTelemetry SDK bypass `BeforeSend`

— my OTel pipeline carries raw client IPs and full SQL text that must never leave the box. A tiny `TraceScope`

dual-writes an `Activity`

and a Sentry span instead, so everything Sentry receives passes my scrubber.

**Breadcrumbs caught my scrubber lying — twice.** A live event's breadcrumb trail contained SQL: EF Core interpolates the query into the breadcrumb *message*, not the structured `data`

bag my scrubber nulled (and my unit tests were green the whole time, asserting exactly the wrong thing). Fixed by dropping EF command breadcrumbs outright — then production found the *same* leak in a second channel: EF logs a failed command at `Error`

level and Sentry's `ILogger`

integration promotes it to an event, SQL in the message again. A scrubber written against one egress path will be bypassed by the next one. Both doors are closed in [#446](https://github.com/mrviduus/textstack/pull/446), and dropping loses no signal — the exception middleware already reports the same failure with the SQLSTATE and constraint name, no SQL.

**Release + environment tags as forensics.** The most interesting issue of the first day showed a dead Ollama starving a metadata pipeline: thirty events, tagged `environment: Production`

. I read it as an outage and started writing the fix. It was my laptop — a dev `.env`

with `ASPNETCORE_ENVIRONMENT=Production`

plus the production DSN I'd pasted in to verify the integration. What broke the spell was Sentry's own metadata: `linux-arm64`

runtime on an x86_64 prod, and a `release`

tag pointing at a commit that had never been deployed. An environment tag is a claim a process makes about itself, not a fact. Now `SENTRY_RELEASE`

comes from the `GIT_SHA`

build arg — every CI-built image has one, no `dotnet run`

ever does — and a Production claim without a release gets renamed `production-unverified`

([#448](https://github.com/mrviduus/textstack/pull/448)).

And the meta-lesson that justified the whole exercise: I verified the integration by sending real events at the real DSN and *reading the captured payloads in the UI* — that's how the leaks, the inferred-geo surprise, and the middleware capture path all surfaced. A monitoring system whose first act is to indict itself is one you can start trusting.

What's the most embarrassing thing your monitoring has ever caught — and was it in the code, or in you?

*I build TextStack, an open-source reader for technical books, in .NET. The full write-up lives on my blog. github.com/mrviduus/textstack*