cd /news/large-language-models/i-built-a-local-llm-rig-to-escape-ap… · home › topics › large-language-models › article
[ARTICLE · art-25856] src=dev.to pub= topic=large-language-models verified=true sentiment=· neutral

I Built a Local LLM Rig to Escape API Bills. Then I Paid OpenAI Again.

A developer running 2asy.ai's filing pipeline built a local LLM rig to escape API costs, but found that OpenAI's batch API outperformed it for large-scale single-document extractions. The local rig remains for live serving and multimodal tasks, while the batch lane moves to OpenAI, achieving 50% cost reduction and zero rate limits.

read1 min publishedJun 13, 2026

I run a one-person AI shop. For 2asy.ai's filing pipeline that needs thousands of single-document extractions per cycle, the local rig lost the batch lane and OpenAI Batch won. Per-pipeline, not per-company.

The rule that decided it: no cross-document attention. Each filing gets its own prompt window. No string concatenation. The rule came from a Neo4j rollback I already paid for.

Quick results.

GGML_CUDA_DISABLE_GRAPHS=1

keeps llama.cpp alive when graph optimizer segfaults.googleapis/python-genai

issue 1984 is not-planned.gpt-5.4-mini ): JSONL line-isolated, 50 percent off, 100-doc nano gate in 2.7 min, zero 429s, around 1 cent per document.The local rig stays for live serving, ER API LLM gate, multimodal, and ablations. The batch lane moves to OpenAI.

Full retrospective with the side-by-side table: https://hannune.ai/blog/local-llm-to-openai-batch.html

source & further reading
dev.to — original article Why AgentTrail Exists: Building Open-Source Audit Trails for AI Agents Uniswap V4 Hooks MEV 2026: Searcher Opportunities and Risks How I stopped dumping PDFs and started chatting with my documentation
── more in #large-language-models 4 stories · sorted by recency
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/i-built-a-local-llm-…] indexed:0 read:1min 2026-06-13 · —