A team moves its nightly agent job onto a free endpoint. Week one looks perfect. Week four, the queue drains ten minutes late and the on-call engineer has no data.
The endpoint did not break. The workload changed shape around it. Free tiers rarely fail loudly; they fail as latency, retries, and quiet queue debt.
This post is a stop rule, not a sales pitch. It gives you one probe, one decision table, and a set of exit criteria you can wire into CI.
Free capacity absorbs small workloads well. It degrades along a curve as concurrency and retries stack up.
Three effects dominate that curve:
None of these raise an alert by default. You only see them if you measure the tail, not the mean.
MonkeyCode is an open-source coding agent project. The operator states that it provides free model access and a free server option, with a free token allowance listed in the terms in effect on 2026-09-15. Quotas and hardware change often, so read the current terms page before you plan around any number.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Free access answers a cost question. It does not answer a fitness question. The rest of this post measures fitness.
You need four measurements before any migration decision:
The script below is a template harness. It has not been executed against MonkeyCode for this article. Adapt the body shape to your provider, then run it in your own environment.
#!/usr/bin/env python3
"""free_tier_probe.py - measure tail latency and error rate on a JSON endpoint."""
import argparse, asyncio, json, os, time
import httpx
PROMPT = "Reply with exactly one word: pong"
async def one_call(client, endpoint, headers, model, timeout_s):
body = {
"model": model,
"messages": [{"role": "user", "content": PROMPT}],
"max_tokens": 4,
}
t0 = time.perf_counter()
try:
resp = await client.post(endpoint, headers=headers, json=body, timeout=timeout_s)
return {"ok": resp.status_code == 200, "status": resp.status_code,
"ms": (time.perf_counter() - t0) * 1000}
except Exception as exc:
return {"ok": False, "status": type(exc).__name__,
"ms": (time.perf_counter() - t0) * 1000}
def percentile(values, q):
if not values:
return float("nan")
ordered = sorted(values)
idx = min(len(ordered) - 1, int(round(q * (len(ordered) - 1))))
return ordered[idx]
async def run(endpoint, api_key, model, total, concurrency, timeout_s):
headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
limits = httpx.Limits(max_connections=concurrency)
async with httpx.AsyncClient(limits=limits) as client:
sem = asyncio.Semaphore(concurrency)
async def guarded():
async with sem:
return await one_call(client, endpoint, headers, model, timeout_s)
started = time.perf_counter()
results = await asyncio.gather(*(guarded() for _ in range(total)))
wall_s = time.perf_counter() - started
return results, wall_s
def summarize(results, wall_s, p95_budget_ms, error_budget):
lat = [r["ms"] for r in results]
errors = [r for r in results if not r["ok"]]
report = {
"requests": len(results),
"wall_seconds": round(wall_s, 2),
"p50_ms": round(percentile(lat, 0.50), 1),
"p95_ms": round(percentile(lat, 0.95), 1),
"max_ms": round(max(lat), 1),
"error_rate": round(len(errors) / len(results), 4),
}
report["verdict"] = "pass" if (
report["p95_ms"] <= p95_budget_ms and report["error_rate"] <= error_budget
) else "exit"
return report
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--endpoint", required=True)
ap.add_argument("--model", required=True)
ap.add_argument("--requests", type=int, default=200)
ap.add_argument("--concurrency", type=int, default=8)
ap.add_argument("--timeout", type=float, default=30.0)
ap.add_argument("--p95-budget-ms", type=float, default=6000.0)
ap.add_argument("--error-budget", type=float, default=0.02)
args = ap.parse_args()
api_key = os.environ["FREE_TIER_KEY"]
results, wall_s = asyncio.run(run(args.endpoint, args.api_key if False else api_key,
args.model, args.requests, args.concurrency, args.timeout))
report = summarize(results, wall_s, args.p95_budget_ms, args.error_budget)
print(json.dumps(report, indent=2))
raise SystemExit(0 if report["verdict"] == "pass" else 1)
if __name__ == "__main__":
main()
Run it with a key in the environment, never in the file:
export FREE_TIER_KEY=... # do not commit this
python free_tier_probe.py \
--endpoint https://<free-endpoint>/v1/chat/completions \
--model <free-model-id> \
--requests 200 --concurrency 8 \
--p95-budget-ms 6000 --error-budget 0.02
echo $? # 0 = keep it here, 1 = start the exit plan
The exit code is the point. A probe that only prints a table gets ignored. A probe that fails CI forces a decision.
Measure cold start separately. Wait at least thirty minutes idle, then time a single call. Record it next to the warm p50.
| Workload shape | Free tier fit | Reason |
|---|---|---|
| Local dev chat, one user | Good | Low concurrency hides tail latency |
| Idempotent lint or test agent | Conditional | Safe only with capped retries |
| Nightly batch inside a wide window | Conditional | Drain time must fit the window |
| Interactive path with a latency SLO | No | No capacity guarantee to cite |
| 10x burst fan-out | No | Retries multiply during bursts |
| Contractual uptime obligations | No | You need a vendor commitment |
Conditional means the probe decides. "No" means no probe will save it.
Write these down while the system is calm. Numbers below are policy examples, not product facts. Set your own.
Criterion five is the honest one. Free compute stops being free when it consumes senior time.
Any two of these means the exit plan should already exist.
Most teams need two of these, not all five.
For everyone else, the probe is cheap and the answer arrives in an hour.
Free access is a legitimate place to start an agent backend. It is a poor place to hide an unmeasured one.
Run the probe against your free endpoint this week. If it passes, keep the workload there and revisit next quarter. If it fails, you now have a dated report to justify the next step, and a soft place to try the free server option before you commit to anything larger.