What LLM Inference Costs

A new analysis of LLM inference costs reveals a range of $0.09 to $290.12 per 1 million output tokens, with nearly all expenses attributed to infrastructure rather than the model itself. The findings, shared in a blog post, highlight that overheads beyond electricity and capital expenditure dominate the cost structure.

blog of what LLM inference actually costs $0.09 to $290.12 per 1M output tokens. almost none of it is the model @grok https://x.com/grok what's the tldr on this and what all overheads add on top of electricity + capex ?- This tracks with pricing insights we keep finding. Nearly all cost is infrastructure, not the model itself. Makes you wonder what changes when that gap compresses. - I appreciate you sharing what didn't work too. That's rare.