{"slug": "ask-hn-what-are-you-using-for-llm-inference-in-production", "title": "Ask HN: What are you using for LLM inference in production?", "summary": "A developer on Hacker News asks what tools others use for LLM inference in production, noting they settled on Gemini 2.5 Flash Lite after starting with OpenAI's GPT-3 and trying other major labs. They express interest in open-source options but have not found comparable cost, performance, and speed, and mention Groq's developer access has been unavailable for months.", "body_md": "| ||||||||||||\n2 points by |\nI started with OpenAI back in the GPT-3 days, then bounced between the major labs (with a brief interlude with Workers AI). I eventually settled on Gemini 2.5 Flash Lite as my workhorse, using it against structured data/vectors as a chatbot. It was cheap, good enough, and most importantly, Groq seems promising but developer access hasn't been available for months. I'd love to use open source, but I never found anything comparable (cost/performance/speed). Would love to hear your suggestions. | |||||||||||\nApplications are open till July 27. |", "url": "https://wpnews.pro/news/ask-hn-what-are-you-using-for-llm-inference-in-production", "canonical_source": "https://news.ycombinator.com/item?id=49121047", "published_at": "2026-07-31 09:46:24+00:00", "updated_at": "2026-07-31 09:52:04.306574+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools"], "entities": ["OpenAI", "Gemini 2.5 Flash Lite", "Groq", "Workers AI", "Hacker News"], "alternates": {"html": "https://wpnews.pro/news/ask-hn-what-are-you-using-for-llm-inference-in-production", "markdown": "https://wpnews.pro/news/ask-hn-what-are-you-using-for-llm-inference-in-production.md", "text": "https://wpnews.pro/news/ask-hn-what-are-you-using-for-llm-inference-in-production.txt", "jsonld": "https://wpnews.pro/news/ask-hn-what-are-you-using-for-llm-inference-in-production.jsonld"}}