cd /news/ai-agents/my-agent-picks-a-free-ai-model-for-y… Β· home β€Ί topics β€Ί ai-agents β€Ί article
[ARTICLE Β· art-170451] src=dev.to β†— pub= topic=ai-agents verified=true sentiment=Β· neutral

My Agent Picks a Free AI Model for You. How Do You Know It's Telling the Truth?

Developer Rudi Tabuti built Fluent Sphere, a single-file, dependency-free Python server that routes prompts across ten simulated free-tier AI providers, ranking models by rating, filtering by capability requirements, and failing over on refusals, scoring 99/100 across three practice trials. Tabuti submitted the agent to the make.fun free-model-aggregator-01 bounty on 15 November 2026, with results due 30 November, and argues that self-reported routing claims cannot be independently verified. "A router you cannot verify is just a black box with good manners," he wrote.

by read9 min views2 publishedNov 16, 2026

By Rudi Tabuti (@bahjoe5)

This article is my entry for the Creator Track of zkVerify Agent Games, Season 01. #AgentGames

My agent routes your prompt to a free AI model and tells you which one answered. It picks the best-rated free model that can handle your request, and if that provider refuses, it fails over to the next one. It does this well: across three practice trials it scores 99 out of 100.

Here is the problem I kept coming back to while building it: every one of those claims is self-reported. The agent says it routed you to the top-rated model. The agent says the first provider refused and it failed over. The agent says the answer came back verbatim. You have no way to check any of it. If the agent quietly routed your prompt to a worse model and labelled it as the best one, you would never know β€” the answer would still look like an answer.

That is the verifiable-agent problem, and it is the reason I am writing this. I built the router first, and then I had to admit that a router you cannot verify is just a black box with good manners.

The agent is called Fluent Sphere. It is a single-file Python server β€” main.py, 1,039 lines, 43,117 bytes, standard library only, no dependencies β€” that speaks two APIs at once:

POST /v1/chat/completions, JSON and streaming (SSE) POST /v1/messages, JSON and streaming GET /v1/models lists the free models it knows about Underneath, it routes across ten simulated free-tier providers: groq, cerebras, gemini, zai, openrouter, sambanova, pollinations, cloudflare, ovh, and kilol. At startup it probes every provider's catalog route (GET <base>/v1/models, Gemini's GET <base>/v1beta/models) and builds the exact catalog it registers, falling back to the ratings table for any provider whose probe fails.

The code is public: https://github.com/raffa-sys/free-model-aggregator-01

I submitted it to the make.fun free-model-aggregator-01 bounty on 15 November 2026 as entry "Fluent Sphere" (https://make.fun/s/01a1dcac-e6f3-76dd-9912-5543146155de). The bounty asked for exactly this: one server, OpenAI- and Anthropic-compatible, routing to the best free model with failover, over ten simulated free-tier providers. Results are due 30 November 2026, so at the time of writing the entry is graded on the dev phase, not yet decided.

One clarification before I go further: the providers in the trials are simulated. The environment stands in for the real providers and answers with the same shapes, the same status codes, and the same refusal behaviour. That makes the routing measurable and repeatable, but it also means nothing here is a claim about a live provider's real-world behaviour.

Every inference request goes through the same three steps: rank, filter, fail over.

Rank. All free models are sorted by rating, best first. Ties are broken by provider id, then model id. The sort key, straight from the code:

out.sort(key=lambda item: (-item[1]["rating_milli"], item[0].id,
                           item[1]["model"]))

Filter. The request's requirements cut the list down before anything is tried: does the model support tools, vision, structured JSON, and does its context length meet the minimum? A JSON-mode OpenAI request may only go to an OpenAI or Gemini wire, because only those can carry the JSON flag; an Anthropic request's JSON requirement is met by any wire, since its JSON need is on the model, not the wire.

Fail over. Candidates are tried in ranked order, and the first 2xx answer wins. The interesting part is what counts as a refusal:

for attempt in range(2):
    status, raw = call_provider(provider, url, request_body, canon["stream"])
    if status == 200:
        result = PARSERS[provider.wire](raw, canon["stream"])
        chosen = row["model"]
        break
    if status == 429:
        continue  # retry once on the same provider
    if status == 402:
        self.dead.add(provider.id)  # budget exhausted: dead for this trial

A 429 (rate limited) gets exactly one retry on the same provider. A 402 (budget exhausted) marks the whole provider dead for the rest of the trial. Hard rejections β€” 400, 401, 5xx β€” also mark the provider dead: the provider answered and rejected this request for good. A network hiccup (no response at all) does not mark the provider dead, because a dropped connection is not a verdict on the provider; the request just moves on to the next candidate. One request makes exactly one successful provider call.

Translation. The request is translated losslessly for everything the client checks: system text, the last user message's text, tool names in order, max_tokens, stop sequences in order, image bytes, and the JSON-mode flag. The provider's answer text and usage counts are relayed verbatim, and the stop reason is mapped back to the client's stop family (stop / length / tool call / refusal). Streaming works the same way: providers stream their answers in 13-character pieces, and the server relays them as SSE frames.

Determinism. Routing is fully deterministic. There is no wall clock and no unseeded randomness anywhere in the decision path, so the same request against the same catalog always takes the same route. That matters more than it sounds: it is what makes the trial scores reproducible instead of lucky.

The bounty kit ships three practice trials, each a workload of 20 requests against the ten providers. All three score 99 out of 100.

The trial score is made of five components, and they are worth spelling out because each one measures a different promise:

Component Weight What it measures
Quality 60 The weighted rating of your valid answers against the best legal rating for each request
Served 15 The share of the 20 requests answered validly
Wire 10 The share of the client's response checks passed
Failover 10 The share of requests whose first provider call refused that you still served
Catalog 5 Whether your --list report matches the provider directory and the free-model sets exactly

GET /v1/models is scored too: asked at the start of the trial with both auth headers, it must list exactly the free model ids you registered.

The quality component sits at 99, which is the practical ceiling. You cannot beat the best legal rating for a request β€” you can only match it β€” so 99 means the router is picking the best eligible model essentially every time, and losing the remaining point to the few requests where the best model is unreachable and the next-best has to do.

The failover component is the one I find most honest about the design. Refusal is not an edge case in this system; it is the normal case. Providers rate-limit you (429) and run out of budget ( 402) in the middle of a trial, and the score explicitly rewards serving the requests whose first provider call refused. The failover rules above exist because the trials punish you if you give up after the first refusal β€” and they punish you again if you keep hammering a provider that has already told you its budget is gone. Marking a 402 provider dead for the trial is what stops the second failure.

The determinism claim is also measurable, not just asserted. Because routing uses no clock and no randomness, replaying a trial against the same catalog reproduces the same routes and the same score. If a score had depended on timing β€” say, on which provider answered first β€” it would not survive a replay. These do.

What I did not measure: anything about live providers. The trials are simulated, the catalogs are fixed, and the ratings are given. The 99 says the router is correct given the ratings and the environment. It says nothing about whether a real provider's free tier behaves the way the simulation says it does.

Here is the uncomfortable part. Everything in the previous section was measured by a harness that I do not control β€” but everything a user of this server would see is reported by the server itself. The routing log, the chosen model, the failover story, the usage counts: all of it comes from the agent's own mouth.

So the honest statement of the system today is: the routing claims are self-reported, and you must trust the agent's log. A user talking to this server has no independent way to check that the model named in the response is the model that actually produced it, or that the top-rated eligible model was even tried first.

That is where zkVerify fits β€” not as a feature I have built, but as the missing piece the design needs. The pattern that fits is TLS attestation, sometimes called zkTLS: a proof that a particular HTTPS response came from a particular server, without revealing the request or the response body. Applied here, it would look like this: the agent returns the answer plus a proof that the provider's API returned a response naming model X. Anyone could verify the proof without seeing your prompt and without trusting the agent's log. The model attribution would stop being a claim and become a checkable fact.

But it is important to say what such a proof would not prove, because overselling verification is worse than having none:

Verification is necessary, not sufficient. It removes one layer of trust β€” the aggregator's word about which model answered β€” but the ranking policy still runs inside the agent, unverified. The stronger design, and the one I would build next, puts the ranking itself inside the proven computation: a proof that the chosen model was the highest-rated eligible model given the public ratings table and the request's requirements. That is a bigger proof, and it is the direction this work points in. The routing layer is the easy part to get right; the proving layer is the actual product.

I want to be explicit about what this is not, because the article would be misleading otherwise:

MAKEFUN_PROMPT_FILE) and a control-plane URL ( MAKEFUN_ENV_URL), registers with the environment, and exits when the workload is done. There is no hosted endpoint you can call today. The next steps, in order, are: host a public endpoint so the router can be used outside the harness; attach a proof of routing β€” starting with attested provider responses, then the ranking itself; and publish the routing log so anyone can replay a request's route from public inputs. Until then, the system is a correct router with an unverified log, and I would rather say that plainly than let the score of 99 do the talking.

Built and written for the Creator Track, zkVerify Agent Games Season 01 (21 October – 21 November 2026). #AgentGames

── more in #ai-agents 4 stories Β· sorted by recency
── more on @rudi tabuti 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain β€” perfect for shipping the agent you just read about.

$git push zahid main
β†’ Live at https://your-agent.zahid.host βœ“
Get free account β†’ Pricing
from €0/mo Β· no card required
LIVE [news/my-agent-picks-a-fre…] indexed:0 read:9min 2026-11-16 Β· β€”