MyVitals extracts lab values with one LLM call, then reconciles raw names against a shared dictionary — exact match, fuzzy match, then an LLM call for the leftovers. If that call fails outright, the pipeline doesn't fail the upload — it degrades: unmatched metrics auto-create a new dictionary entry instead of finding the right one. Good design, on its own.
Here's the problem: an env var meant for the OCR endpoint got set on the variable that goes to Chat Completions instead. Every metric-matching call, for about a month, hit 400 Invalid Model. Every single one. And because of the graceful degradation above, every upload still reported success.
The visible symptom wasn't an error — it was a canonical-metrics dictionary slowly fragmenting into near-duplicates. Nothing about that looks like a bug from the outside. The metric everyone watches, "did uploads succeed," stayed at 100%.
Lesson: when a stage is designed to degrade instead of fail loudly, the success of the operation it's embedded in tells you nothing about whether that stage ran. You need a metric for the stage itself.
Not an alert, not a test — there's no integration harness exercising the real pipeline. A routine cost audit noticed the LLM-matching cost line had gone quiet, which prompted someone to actually read the error logs. The diagnosis — 400 Invalid Model — had been sitting in the database the whole time, inside a raw error string, in a tooltip nobody had reason to open.
The fix wasn't more data, it was structuring what already existed: typed failure classification (error type, HTTP status, retryability) grouped by that classification instead of raw message text. 4× · 36% of matching calls · invalid_model is a bug report. Four chronological stack traces is a wall nobody reads.
Extraction runs synchronously inside the upload request — no job queue yet. Nginx's proxy_read_timeout is 120s. If retry logic doesn't know that number: three 90-second retries run for 4.5 minutes against a socket Nginx already closed. Client gets a 504, server keeps working on nothing, report sits at "processing" forever.
Fix: a shared wall-clock budget (105s, deliberately under 120s) passed through both extraction and matching, so retries stop before the proxy would kill the connection anyway. Retries only happen for genuinely transient failures — a 401 isn't going to succeed on attempt two, and every retry is a real charge. A provider's Retry-After is honored as sent, not clamped — ignoring a rate limiter's instructions just earns another 429 faster.
Your retry budget and your proxy's timeout are the same constraint wearing two config files. Tune only one and you haven't fixed anything — you've just moved where the zombie request dies.
👉 Try MyVitals now — the pipeline now tells on itself when something's wrong.