Blocked Time Makes Free Capacity the Wrong Bet

A developer argues that free LLM inference lanes are often the wrong choice because the real cost is occupied time, not token price. The writeup proposes a small Python break-even model that compares token cost plus blocked-time cost between a free lane and a paid lane, and recommends timing each call to capture queue and inference latency. Overnight batch jobs can still rationally use free capacity, but work that blocks a reviewer, test gate, or deploy usually cannot.

Free capacity is the wrong bet when you are blocked on the answer. Token price is a sticker on the box. The unit you actually pay is the time you cannot move. You already feel this in the editor. You send a prompt, the free lane queues, and you hover because the next edit depends on the output. That hover is not leisure. It is occupied time wearing a cheaper costume. Most cost notes start with price per million tokens. The number is easy to paste into a spreadsheet and easy to lie with. A free lane can win the spreadsheet and still lose the job, because the job is not a token. The job is a blocked reviewer, a blocked test gate, or a blocked deploy. Picture an express checkout that charges nothing and does not move. You would not stand there with melting ice cream because the sign said zero. Inference queues are that lane. The ice cream is your attention, and it melts in minutes, not billing cycles. Run a rule before you park work on free model access or a free server. Estimate how long you will be blocked. Multiply by what that blockage costs. Compare the product to the cash premium of a lane you control. If blockage costs more, free is the wrong bet. Overnight batch work can sit in a free queue and still be rational. Occupied work cannot. The distinction is occupancy, not urgency. A job with no deadline can still be the wrong free-lane bet if you refuse to context-switch. A job with a deadline can still be a good free-lane bet if it can sleep until morning. You are not pricing the model. You are pricing whether the clock is allowed to own you. Keep the accounting small enough to sit next to the client. Treat the code below as a proposal you run against your own waits. It is not a vendor benchmark, and the dollar figures in the worked example are labeled fiction. proposal: occupied-time break-even for a single LLM job from dataclasses import dataclass @dataclass class Attempt: queue s: float infer s: float prompt tokens: int completion tokens: int ok: bool @dataclass class Lane: name: str usd per mtok: float 0.0 for a free lane loaded usd per hour: float you, CI, or the blocked pipeline def job seconds attempts: list Attempt - float: return sum a.queue s + a.infer s for a in attempts def token usd attempts: list Attempt , lane: Lane - float: toks = sum a.prompt tokens + a.completion tokens for a in attempts return toks / 1 000 000.0 lane.usd per mtok def occupied usd attempts: list Attempt , lane: Lane - float: return job seconds attempts / 3600.0 lane.loaded usd per hour def total usd attempts: list Attempt , lane: Lane - float: return token usd attempts, lane + occupied usd attempts, lane def free is wrong bet free attempts: list Attempt , paid attempts: list Attempt , free lane: Lane, paid lane: Lane, - bool: """True when occupied time makes the free lane more expensive.""" return total usd free attempts, free lane total usd paid attempts, paid lane Charge every retry to the same job. A 429, a truncated completion, a tool call you have to repeat — each one buys another queue ticket and another slice of occupied time. If you only price the happy path, free capacity always looks like a gift. The gift is the first attempt. The bill is the tree. Wire a clock around the call so the numbers are yours. python import time from urllib.error import HTTPError def timed complete call, prompt: str - Attempt: t0 = time.monotonic t queue end = t0 try: Label: wrap your real client here. This is a sketch, not an SDK. t queue end = time.monotonic result = call prompt t1 = time.monotonic return Attempt queue s=max 0.0, t queue end - t0 , infer s=t1 - t queue end, prompt tokens=int getattr result, "prompt tokens", 0 , completion tokens=int getattr result, "completion tokens", 0 , ok=True, except HTTPError: t1 = time.monotonic return Attempt queue s=max 0.0, t queue end - t0 , infer s=t1 - t queue end, prompt tokens=0, completion tokens=0, ok=False, If you cannot separate queue wait from inference, do not fake the split. Dump the whole elapsed time into queue s and set infer s to zero. Over-attributing to the queue is safer than pretending the model was slow when you were standing in line. Honesty beats a pretty flame graph you invented. Here is a worked example. These waits are illustrative. Do not read them as a measurement of any public host. Suppose your loaded cost is 80 USD per hour, a fully loaded engineer or a CI minute that blocks a release train. The free lane needs two attempts: 90 seconds queued and 8 seconds of inference, then a retry of 70 seconds queued and 8 seconds of inference. Tokens are free. The paid lane needs one attempt: 3 seconds queued, 8 seconds of inference, at 0.80 USD per million tokens for a 4k-token round trip, which is a rounding error. free lane = Lane "free", usd per mtok=0.0, loaded usd per hour=80.0 paid lane = Lane "paid", usd per mtok=0.80, loaded usd per hour=80.0 free attempts = Attempt queue s=90, infer s=8, prompt tokens=2800, completion tokens=1200, ok=False , Attempt queue s=70, infer s=8, prompt tokens=2800, completion tokens=1200, ok=True , paid attempts = Attempt queue s=3, infer s=8, prompt tokens=2800, completion tokens=1200, ok=True , print round total usd free attempts, free lane , 2 3.91 print round total usd paid attempts, paid lane , 2 0.25 print free is wrong bet free attempts, paid attempts, free lane, paid lane True Just under three minutes of occupied time at 80 USD per hour is about 3.91 USD. The paid lane is about 0.24 USD of your time plus a fraction of a cent in tokens. Free lost by more than an order of magnitude, and nobody missed a hard deadline. You were simply not free to context-switch. That last clause is the switch. If you can drop the prompt and go write tests, your loaded cost for that wait collapses toward zero. Free capacity becomes a good bet again. The break-even is not a property of the model. It is a property of whether you are blocked. Retries change the slope. One extra 429 on the free lane adds another full queue you do not control. One extra 429 on a lane you control adds a backoff you chose. If your client retries three times by default, you are not comparing one free call to one paid call. You are comparing two wait trees, and the free tree grows faster because you do not own the queue discipline. Budget the retry path as if it were the job. In occupied work, it is. CI makes the same shape with different clothing. A runner minute can look cheap on the invoice while the release train behind it is expensive. If six people are waiting on a red gate, your loaded usd per hour is not one salary. It is six people pretending to check chat. Put the larger number in, or stop claiming the free lane is saving the company money. You can watch the clock without a dashboard. proposal: time a single completion from the shell replace the client; do not treat the duration as a vendor claim /usr/bin/time -f 'wall s=%e' your-complete --prompt-file ./job.txt --out /tmp/out.txt Run it three times on a quiet afternoon and three times at peak. Keep the worst quiet run and the worst peak run. Feed both into free is wrong bet . If peak already loses, stop sending occupied work to that lane during peak. If even quiet loses, the lane is a batch drain, not an interactive tool. Write the decision next to the client, not in a wiki you will not open while the spinner spins. A tiny check keeps the function from rotting into folklore. proposal: pin the inequality so a later refactor cannot "save" a blocked job def test occupied free lane loses : assert free is wrong bet free attempts, paid attempts, free lane, paid lane def test unoccupied free lane can win : idle = Lane "free", usd per mtok=0.0, loaded usd per hour=0.0 paid = Lane "paid", usd per mtok=0.80, loaded usd per hour=0.0 assert not free is wrong bet free attempts, paid attempts, idle, paid The second test is the point. Zero occupancy, free wins. Nonzero occupancy, you have to do the math. Do not ship a culture that treats those two cases as the same policy. Where does a free product lane fit this picture? MonkeyCode offers free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Use that kind of lane for the accounting script, for overnight evals, and for prompts you can drop on the floor. Do not use it as the blocking step inside a review, a release, or a customer call. The free server is a lab bench. It is not a reserved checkout. A lab bench is still useful. You can point the same timed complete wrapper at a scratch project, collect Attempt rows, and learn your own occupancy number. That is the honest job of free capacity: it teaches you the shape of the queue before you put money, or a human, on the line. Measuring on a free lane is a good bet. Blocking on a free lane is how you donate an afternoon and call it thrift. Limitations, said plainly. This method overcharges if you actually context-switch. It undercharges if a blocked job stalls three other people who are not in your hourly rate. It ignores quality. A free completion that you rewrite by hand is another occupied loop, and the script will not see it unless you record the rewrite as another attempt. It also ignores fairness. Hammering a shared free server because your break-even said "go" is how shared lanes die for everyone else. The formula does not give you a moral claim on someone else's spare capacity. Who should not use this approach. Do not use it to justify unpaid overtime. Do not use it as a performance claim about any hosted model. Do not use it for safety-critical or secret work on a shared free server. Do not use it if your loaded cost is a vanity salary number you have not agreed with your team. Garbage in, theatrical out. And do not use it as a reason to skip the paid lane when the person waiting is a customer. Their occupancy is not in your spreadsheet, which is exactly why it belongs in the decision. If your work can sleep, let it sleep on the free lane. If your work is staring at you, pay for a lane you can schedule, or do the step without a model. The clock owns the job. The token price is just the caption. Time your own waits, then decide which bet you are actually making.