Free Tokens Are Not an SLO: An Ops Cost Drill for AI Batch Queues

MonkeyCode's product outreach demonstrates that free AI model tokens are not an SLO, using a cost drill for batch queues that converts per-token bills into per-hour operational costs. The drill shows that even with zero token cost, a 1,000-request job can incur $24 in human time under normal conditions, and a three-minute network fault can push deadline slack negative, emphasizing the need to monitor retry ratio and deadline slack as control thresholds.

This week, two numbers trended: a harness at 100%, a model at 30%. For platform teams, a better pair is queue age and deadline slack. This article is a cost drill for the simplest AI batch path: free tokens, free server, non-negotiable deadline. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. That capacity is real. It is not an SLO. The tokens cost nothing. The queue is patient. Your deadline is not. Token cost is easy to measure. Operations cost is easy to ignore. A free endpoint converts a per-token bill into a per-hour bill. The bill becomes your time, your retries, and your queue age. This drill keeps the ledger honest. It answers one question: what does a completed request cost when the token price is zero? python worker.py minimal, single-threaded import queue import time import csv work = queue.Queue for i in range 1000 : work.put {"id": i, "prompt tokens": 512, "max tokens": 256} def call model payload : replace with your free model endpoint return {"ok": True, "in tokens": 512, "out tokens": 180} completed = 0 retries = 0 started at = time.time while not work.empty : item = work.get attempt = 0 while attempt < 4: try: call model item completed += 1 break except Exception: retries += 1 attempt += 1 time.sleep 2 attempt The worker is deliberately single-threaded. Free capacity often serializes. Serialization turns a token problem into a time problem. python cost ledger.py import csv import time HOURLY OPS COST = 50.0 loaded engineering rate, adjust def record item, elapsed, retries : with open "ledger.csv", "a", newline="" as f: csv.writer f .writerow item "id" , round elapsed, 3 , retries def report completed, retries, elapsed s, deadline s : ops cost = elapsed s / 3600.0 HOURLY OPS COST retry ratio = retries / max 1, completed slack = deadline s - elapsed s print f"completed={completed}" print f"retries={retries}" print f"wall clock s={elapsed s:.1f}" print f"ops cost usd={ops cost:.2f}" print f"retry ratio={retry ratio:.3f}" print f"deadline slack s={slack:.1f}" return retry ratio, slack The token spend is zero. The ledger rows still carry a cost. A row looks like this: id,elapsed s,retries 0,1.234,0 1,3.456,2 High retries on early rows mean throttling, not a crash. Growing elapsed times mean the queue is the bottleneck. Both are signals for one control decision. Normal free-tier conditions: | Metric | Expected value | |---|---| | Completed | 987 / 1000 | | Retries | 214 | | retry ratio | 0.217 | | wall clock s | 1742 29:02 | | ops cost usd | 24.19 | | deadline slack s | 58 | That is the good case. 58 seconds of slack. 24 dollars of human time on a "free" job. Cut the network to the endpoint for three minutes. tc qdisc add dev eth0 root netem loss 100% The worker retries with backoff. The queue grows. The ledger fills. After the fault: | Metric | After injection | |---|---| | Completed | 801 / 1000 | | Retries | 1034 | | retry ratio | 1.291 | | wall clock s | 1900 | | ops cost usd | 26.39 | | deadline slack s | -20 | Negative slack. Deadline gone. Tokens still free. Read the ledger as a control loop. Two thresholds matter more than token spend: retry ratio 0.10 deadline slack s < 0 When either fires, stop the worker and re-route. kill $ pgrep -f worker.py export MODEL ENDPOINT="https://paid.example/v1" ./worker.py --resume ledger.csv The ledger turns a crash into a resume. You know which items completed. tc qdisc del dev eth0 root kill %1 rm -f worker.py keep ledger.csv if you want a cost trend This drill assumes you control the client. It does not measure shared CPU noise, hidden rate limits, or model quality. Those need separate experiments. Do not use this pattern for regulated data, payments, or any job with a hard SLO. The paid path exists for a reason. Free capacity is a bet. The ledger is the odds table. Run the drill, set the thresholds, and stop before the deadline sign turns negative.