What I got wrong estimating LLM costs before the request
A developer built a cross-provider LLM gateway that refuses requests before they are sent when today's spend plus in-flight reservations plus an estimate would exceed a user-set daily cap, returning HTTP 402 instead of l…