The self-hosting math for LLMs quietly inverted this year

A developer's analysis of self-hosting economics for AI models reveals that the cost calculus has shifted dramatically in 2025, with API token prices falling while GPU hardware costs have risen sharply. The developer argues that for many narrow tasks, running small open-weight models on modest hardware is more cost-effective than frontier models, and that organizations should route workloads based on data sensitivity and task complexity rather than assuming self-hosting is cheaper.

Most of the self-hosting advice you'll find was written against 2024 assumptions. API pricing is expensive and linear, hardware is fixed and amortizes, so past some volume you break even. Buy the GPU, stop paying rent. Two things broke that this year, and they moved in opposite directions. OpenAI cut GPT-5.6 pricing twice over the summer, dropping Luna by 80% and Terra by 20% at the end of July, then Sol by over 20% in August. Claude Haiku 4.5 runs $1 per million input tokens and $5 per million output, with up to 90% off cached input. Normal continuation of a multi-year trend. Nothing surprising. The NVIDIA RTX PRO 6000 Blackwell showed up in early US retail around $8,565 in March 2025. NVIDIA's Marketplace price hit $13,250 in June 2026, then $16,000 in August. Roughly 87% above launch, same silicon, driven by the memory shortage rather than anything about the card. So if you're amortizing hardware over three years, you're betting a falling price curve stays above a fixed cost for 36 months, except the fixed cost isn't fixed either. Replacement and expansion both got more expensive while you weren't looking. Here's one real query from my production telemetry. Research task on Haiku 4.5 with web search enabled: | Component | Quantity | Cost | |---|---|---| | Input tokens | 28,965 | $0.0290 | | Output tokens | 3,097 | $0.0155 | | Web search calls | 3 | $0.0300 | | Cached tokens | 0 | $0.0000 | | Total | 32,062 tokens, 57s | $0.0745 | Two things in there matter more than the total. Forty percent of that bill wasn't the model. Three tool calls at a cent each cost more than the input tokens did. Every cost comparison you'll read argues token price, which is the half that's been falling. Cached tokens were zero. Most of that 29k input is a system prompt that barely changes between runs. Prompt caching would cut a large share of it. There was a cheaper optimization sitting right there, before any hardware conversation. Before you model anything, pull thirty days of usage and separate token cost from tool cost. Your request shape is probably not the one in whatever blog post you're reading. There's a persistent assumption that self-hosting means running a 70B model. Qwen's lineup doesn't have that rung. The open-weight local tier is 27B dense, with Qwen3.8-27B on Hugging Face since August under Apache 2.0. Above that it jumps to Qwen3.8-Max at 2.4T parameters as a sparse MoE. Reported memory for the 27B: roughly 56GB at BF16, about 28GB at FP8, and 14 to 17GB at 4-bit, plus KV cache. The 262k context window means budget real headroom above the weight figure. Treat those as reported, not measured. Pull the quant and watch actual resident VRAM at a realistic context length before anyone signs a purchase order. But the shape is clear: the local tier for narrow tasks runs on a 24GB card, not a $16,000 workstation card. Those were never the same purchase. Document classification, PII redaction, entity extraction, and ticket routing are not frontier problems, and a small model fine-tuned on your domain frequently beats a general-purpose giant on exactly that kind of narrow repetitive work. Most PHP shops have historically had one option for AI work: bolt on a Python microservice and fragment the team. Neuron AI https://github.com/neuron-core/neuron-ai is an agentic framework for PHP 8.1+ that keeps orchestration in the existing runtime. Multi-agent, RAG, structured outputs, built-in observability. The useful part for a mixed architecture is that swapping providers is a config change rather than a rewrite, which is the actual hedge against both lock-in and price moves. Not owning the hardware. Being able to move. Don't self-host to save money right now. The token price is falling and the hardware price is climbing, so that trade is worse than it was eighteen months ago. Self-host what can't leave the building, route the rest to an API, and size the local side to the task rather than the frontier. The hard part isn't the infrastructure. It's classifying which workloads are which, and that has to happen before anyone writes code. Longer version with the compliance angle, the confidential computing middle path, and a decision table: Self-Hosted LLMs vs API Providers: Why the Math Changed https://www.zadrosolutions.com/insights/self-hosted-llm-vs-api