A team has one free AI lane and fourteen candidate workloads. The intuitive allocation is by seniority or by who asked loudest. That rule usually hands the lane to the noisiest team rather than the safest workload, and the first bad night ends the experiment for everyone.
The alternative below is an assignment sheet, not a procurement doc. It ranks workloads by two measurable axes — burst ratio and blast radius — then applies four hard gates before anyone gets access. Everything here is a decision aid; it is a conversation tool, not objective truth.
Scenario note: the numbers in the tables are illustrative placeholders, not measurements from a real team or product.
| Variable | Definition | Where it comes from |
|---|---|---|
B — burst ratio |
p95 daily requests ÷ p50 daily requests for the same workload | Your own gateway, proxy, or app logs |
H — revert hours |
Wall-clock hours to undo the worst plausible bad output of one run | The owning team's honest estimate, written down |
F — fallback |
A second path that works without the free lane | Owner names it; someone else verifies it once |
Two thresholds, chosen by you and written down before scoring: B_max and H_max. A team that cannot name B_max in advance will rationalize it after seeing the scores.
The script below is illustrative and was not executed for this article. It assumes you can export a CSV with date,workload rows.
import csv, statistics
from collections import defaultdict
per_day = defaultdict(lambda: defaultdict(int))
with open("ai_requests.csv") as f:
for row in csv.DictReader(f):
per_day[row["workload"]][row["date"]] += 1
def burst_ratio(daily_counts):
if len(daily_counts) < 2:
return None # not enough days to say anything
p50 = statistics.median(daily_counts)
if p50 == 0:
return None # sparse workload: percentile ratio is noise
p95 = statistics.quantiles(daily_counts, n=20)[18]
return round(p95 / p50, 2)
for workload, counts in sorted(per_day.items()):
print(workload, burst_ratio(list(counts.values())))
Prerequisites and an honest check:
None as "insufficient evidence", not as a low score. Sparse workloads need two weeks of shadow logging first.
Illustrative scores for four of the fourteen candidates:
| Workload | p50/day | p95/day | B |
H |
F named? |
Gate result |
|---|---|---|---|---|---|---|
| Changelog summarizer | 12 | 14 | 1.2 | 0.5 | Yes (manual edit) | Pass — first in queue |
| Test-name suggester | 30 | 33 | 1.1 | 1 | Yes (skip step) | Pass |
| Release-note generator | 20 | 140 | 7.0 | 2 | No | Hold — fails B andF |
| Migration script reviewer | 6 | 6 | 1.0 | 48 | No | Reject — fails H regardless of burst |
Note what the sheet does to the loudest case. The migration reviewer is the highest-stakes workload and the lowest-volume one; a burst-based argument would wave it through, and H = 48 blocks it anyway. That is the point of scoring both axes.
Apply in order. A workload that fails any gate leaves the queue; it does not "go to the end".
Q be the free capacity you actually observe for your workload, confirmed in the product console rather than assumed. Require p95/day ≤ 0.7 × Q. The 0.7 is your margin, not a product limit — pick it and write down why.H ≤ H_max. If a bad run cannot be undone inside that window by the owning team, the workload is not a free-lane candidate.
Escalate a workload off the free lane when any of these holds for two consecutive review periods:
p95/day > 0.7 × Q — headroom gate about to failF was exercised during themH grew because the workload's outputs now touch something harder to revert
The third one is the quiet failure. A workload can stay well inside capacity while becoming far more expensive to revert, and the capacity number will look fine the whole time.
Free model access and the free server option together mean the lane can be a place to run a job, not only a model endpoint to call. That shifts H in a way model-only access does not: generated artifacts, intermediate state, and any scheduled jobs can live on that server, so a bad run may leave residue beyond a bad response.
Ask two extra questions before assigning a workload to the server option:
If the answers are "yes" and "not sure", the workload is not ready for the server option even if it passes gates 1, 2 and 3.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is an open-source project offering free model access and a free server option for teams to try; current limits, available models, and environment details should be read from its own documentation before you set Q, because those specifics change and none are asserted here.
B; log in shadow mode first. B_max, H_max, and your margin once, together, before scoring.
If you only take one thing from this: a free lane goes to the workload that is cheap to reverse and steady in shape, and it keeps going to that workload only until the numbers say otherwise.
Which would reverse your allocation first — a burst ratio above B_max, or a revert estimate higher than the team expected? That answer is the variable worth logging next month.