cd /news/ai-infrastructure/we-re-launching-our-llm-gateway-with… · home topics ai-infrastructure article
[ARTICLE · art-125661] src=dev.to ↗ pub= topic=ai-infrastructure verified=true sentiment=↑ positive

We're launching our LLM gateway with observability, evaluators and git-like prompt management.

A team of developers has launched Infere, an OpenAI-compatible AI routing and observability platform, now in beta. The gateway selects the appropriate model per request, versions prompts like code, scores them with evaluators, and logs every call to track costs. The team is inviting developers to stress-test the platform against real traffic and provide feedback.

by read1 min views2 publishedSep 10, 2026

We've spent the last few months building Infere, an AI routing and observability platform, and today we're putting it in front of real developers. We'd love for you to be the ones to stress-test it.

It started the way these things usually do, we were annoyed. Every AI feature we built meant another provider SDK, another auth quirk, another error format, another dashboard we'd have to open to find out what anything cost. Switching a model was a sprint. Nobody could answer "what did the support bot cost us this month" without a spreadsheet and a guess.

So we built Infere: one OpenAI-compatible endpoint we point our apps at, with a layer around it that picks the right model for each request, versions prompts like they were code, scores them with evaluators, and logs every single call so the cost is never a mystery again.

Point it at our endpoint, and routing, budgets, and observability kick in on your first request. Prepaid credits, pay as you go, credits never expire.

We're not going to pretend it's finished. We're still in beta. But it works, it's live, and we want people to actually use it against their real traffic, that's the feedback we can't invent.

Try it, break it, and tell us what's missing from the request path. We read everything.

── more in #ai-infrastructure 4 stories · sorted by recency
── more on @infere 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/we-re-launching-our-…] indexed:0 read:1min 2026-09-10 ·