Two weeks ago I had zero dollars, one gaming PC with an RTX 3060 Ti, and a stubborn idea: what if the cost of running a small, useful LLM-backed API could be close to zero, if you just... hosted the model yourself?
Here's what I shipped, and the specific things that broke along the way — because the failures were more instructive than the plan.
A small API that turns plain English into working code artifacts:
/v1/regex
— "validate a Brazilian CEP" → a working regex, an explanation, and match/no-match examples/v1/sql
— "list the 10 customers who bought the most last month" → a SQL query/v1/commit-message
— a diff description → a conventional commit message/v1/json-schema
— "an e-commerce product with name, price, category" → a JSON SchemaNothing revolutionary. The point wasn't the idea — it was proving the economics work when the model is yours.
qwen2.5-coder:7b
locally on a Windows box with an RTX 3060 Ti (8GB VRAM — plenty for a 7B coder model at Q4 quantization){...}
block if json.loads
fails on the raw text)Total infrastructure cost: $0. The GPU was already sitting there.
Windows process persistence is not what you think. Anything you launch directly in an SSH session — even with Start-Process
, even backgrounded — dies the moment the SSH session closes, because Windows ties it to a Job Object scoped to that session. The fix that actually works: wrap the command in a .bat
, register it as a Scheduled Task (schtasks /create ... /sc onlogon
), and trigger it once immediately with schtasks /run
. That survives disconnects.
"Free static domain" isn't always what it says. I tried ngrok's free static domain (a real, permanent feature — not a myth) to get a stable URL instead of Cloudflare's rotating one. It works great in a browser. It's useless for an API: ngrok's free tier shows a mandatory interstitial warning page to any request that doesn't send a specific ngrok-skip-browser-warning
header — and a marketplace proxy calling your API on a customer's behalf will never send that header. Every single API call returns an HTML warning page instead of JSON. I found this by testing the raw endpoint externally with no special headers before trusting the "solved" checkbox — which is the actual lesson: test the exact path a real client will take, not the happy path you control.
LLMs don't reliably return valid JSON, even when told to. The fix wasn't a smarter prompt — it was a tolerant extractor: try json.loads
on the raw text, then try pulling a fenced
block, then fall back to a brace-matching regex. Three tries, cheap, and it turned "the model sometimes wraps JSON in markdown" from a 502 error into a non-issue.
**A brand-new marketplace listing needs its own free tier.** I gated the public demo (`/demo/*`) with a simple in-memory per-IP daily counter (5 free tries/day) instead of requiring signup at all for the landing page. Letting people try before they subscribe converts better than a wall.
## The economics, honestly
RapidAPI takes 25% of marketplace revenue, plus a small payout processing fee. That's real, and it means the math on a $9.99/month plan is closer to $7 net than $10. Worth knowing before you price.
## What's next
The backend is done. The actual bottleneck now is distribution — a working API with zero users doesn't pay rent. If you're building something adjacent (dev tools, LLM-backed APIs, or you're just curious about the self-hosted-model economics), I'd genuinely like to hear what you're seeing.
**API listing:** [https://rapidapi.com/JoaoPauloNA/api/plain-english-to-code-api](https://rapidapi.com/JoaoPauloNA/api/plain-english-to-code-api) (free tier, no credit card to try the demo)
---
*Building this in public as I go — happy to answer questions about the self-hosted LLM economics, the FastAPI/Ollama integration, or the Windows automation quirks above.*