I’ve been using the Claude and GPT APIs for various projects, but the monthly bills are starting to get out of hand—they range from about $100 to $200 per month, depending on usage. So I’m considering switching to a self-built, on-premises inference system.
The initial hardware cost is about $2,000. Based on my current API usage, I’d recoup that cost in about a year. But I’m not entirely sure if an on-premises model can truly match the performance of the cloud for the tasks I’m working on.
I forgot to mention that the web version worked fine for me before—it was just that I ran out of quota. Do you think I need to set up an on-premises inference system?
source & further reading
forum.level1techs.com — original article
Budget GPUs or use spare 4090 for local model? Use case included.
Wait for RTX Spark Laptop or get a DGX Spark (or other GB10 like Asus Ascent, etc) HELP!
Bang for buck AI performance