Lists of cost-saving tips are ordered by how interesting the tip is. The order you want is by money saved per hour of work, and that order is different for every bill — so here is the list, and here is the arithmetic that sorts it.
A lever is worth doing in proportion to three things and inversely to one:
score = (B * f * r) / (h * w)
B monthly bill, in dollars
f fraction of B this lever touches (0..1)
r reduction it achieves on that portion (0..1)
h engineer-hours to build it
w ongoing hours per month to keep it alive, +1 so w >= 1
The numerator is dollars saved per month; the denominator is the cost of having done it. Score is dollars per hour, so it is comparable across levers and comparable against whatever else that engineer would have built. Two rules make it behave: f
and r
are estimates, so round them to the nearest 0.1 and do not pretend otherwise; and w
exists because a clever routing layer that needs an hour of babysitting a week is not free, however good its r
is.
One worked score, entirely from assumed inputs: a $4,000/month bill, prompt caching applied to the 80% of spend that has a stable prefix, achieving a 40% reduction on that portion, costing 6 hours to implement and about an hour a month to maintain. score = (4000 × 0.8 × 0.4) / (6 × 2) = 1280 / 12 = $107 saved per hour of work
. Compare that with a model-swap evaluation that touches 100% of spend at a 25% reduction but costs 40 hours of evaluation: (4000 × 1.0 × 0.25) / (40 × 1) = $25/hour
. Both are worth doing; only one of them is worth doing this week.
These change what work happens at all. They have the largest r
and the largest h
.
These leave the architecture alone and shrink the terms in the per-request formula. Small h
, and unusually good scores.
n
pays for all n−1
before it. A sliding window plus a rolling summary turns that back into a linear cost.h
close to zero.Fill in f
and r
from your own per-request model, guess h
honestly and let the formula sort. Three things fall out of it almost universally, and they are worth naming because they are counterintuitive:
h
win even at unimpressive r
. A 10% reduction that takes an hour beats a 50% reduction that takes a fortnight, on this month’s numbers.f
near zero cannot be worth doing however elegant it is. Check what fraction of the bill you are aiming at before you start.w
should be viewed with suspicion. The ongoing cost is paid forever and the saving usually is not.Re-run the sort after every change. Each lever you pull alters B
and the f
of everything else, and the second-best lever on the original list is frequently not the second one you should do.
Lever 11 needs a price-per-provider view of the same model to be actionable at all — the provider listing shows which providers serve a given model and at what price, which is the input that turns “route on price” from a principle into a config value.