AI agent cost drift is the slow growth of your input floor — system prompt, tool schemas, CLAUDE.md, MCP servers — that a rolling baseline never catches, because the baseline climbs with it. drift_anchor_gate.py
pins a frozen canary on day 0 and compares: a 0.35%/day creep raised zero alarms in 60 days; the anchor blocked it on day 9.
Your fleet dashboard fires when today's average run jumps 20 percent above last week. Your context floor grows 0.35 percent a day. Those two numbers never meet. I built six 60-day worlds and ran four rolling detectors over them, each at the tightest threshold that stays quiet on a flat fleet, and the slow creep raised zero alarms in all four, across 60 days, while the floor went up 22.6 percent. A frozen anchor caught the same creep on day 9.
AI disclosure:I wrotedrift_anchor_gate.py
andmake_worlds.py
with an AI assistant and ran them myself: Python 3.13.5, offline, standard library only, no network, no keys. Every number, exit code and sha256 below is pasted from a real local run. I ran the whole demo twice from a cleanrm -rf worlds
, and the twooutput.txt
files are byte-for-byte identical (sha256 1772e695cb75f79d9e3f162ed4c49477a329703610cdf5ddff54cec2cc4da62a
). The series are synthetic, and I say exactly how they are generated below. The one thing that is not synthetic is the arithmetic, and it is the part that does the work.
In short:
w
firing at ratio T
cannot see any uniform daily growth below g* = T^(2/w) - 1
. For w=7, T=1.20
that is diff -rq
prints nothing, git diff
would print nothing), the vendor quietly bumped the harness scaffold, the fleet dashboard's short-window detectors stayed silent (the long window fired once, on day 58 — 46 days late), and the anchor blocked on day 12 with the growth attributed to vendor: +2600 B
.On July 13, Dipankar Sarkar replied to my post on usage logs that report zero tokens for delivered text with a sharp correction. His argument: a provider tokenizer bump and real suppression look identical if you only watch a threshold, so the disambiguator should be population, not threshold. "So the block condition isn't 'delta widened past X,' it's 'this stream's delta diverged from the fleet's,' per-stream anomaly against a rolling baseline of everyone else."
He is right, and I said so. Then I argued the thing that became this article: a rolling baseline adapts, so anything that grows slower than the adaptation window never diverges from the fleet. It walks the fleet along with it. Boiling frog. What you need is a second anchor that is frozen at a known-good date, and for a solo developer with no fleet, a canary request with deterministic input is a population of one.
And then I ended my own comment with this:
"To be straight about status, this is a design argument and not something I have measured. The re-tokenized floor I have run. The fleet-versus-canary discrimination I have not."
That was two days ago. This post is me paying that off. The domain moved (I am measuring input cost drift, not usage suppression) but the structure of the question is the same one Dipankar and I were arguing about: can a baseline built from your own recent history see a thing that moves your own recent history?
This part is not a discovery. It is a derivative, and it is worth doing on paper before you touch a fixture, because it tells you what the measurement is allowed to find.
A rolling window compares today against a baseline built from the previous w
days. That baseline is centred roughly w/2
days back. Under uniform daily growth g
, today's level over the baseline's level is about (1+g)^(w/2)
. The alarm fires when that ratio exceeds the threshold T
. Solve for g
:
g* = T^(2/w) - 1
Any growth slower than g*
is invisible forever. Not "for a while". The ratio the detector computes stops rising, because the baseline is climbing at the same rate as the signal. Here is what the tool prints before it has read a single byte of data:
window threshold invisible below compounds to, over 30 days
w=7 T=1.15 4.07%/day x3.31
w=7 T=1.20 5.35%/day x4.77
w=14 T=1.20 2.64%/day x2.18
w=28 T=1.20 1.31%/day x1.48
w=28 T=1.10 0.68%/day x1.23
Read the right-hand column. A seven-day window at a 20 percent threshold is blind to any drift that multiplies your input cost by 4.77x over a month. That is the prediction. The measurement is not the formula, it is the question of where real context creep lands relative to that band, and whether you can pick a T
low enough to close it without the alarm screaming every Tuesday.
Sixty days. Forty runs per day, fixed. I fixed the run count on purpose: variable traffic adds noise to a daily average, and noise is the thing that pushes the dashboard's threshold up. Removing it is a handicap in the dashboard's favour, and I would rather hand the dashboard its best case than be accused of rigging it.
The floor at pin time is 61,060 bytes:
| field | bytes | origin |
|---|---|---|
tools_json |
||
| 26,000 | local file | |
system_prompt |
||
| 12,000 | local file | |
instructions (CLAUDE.md) |
||
| 9,000 | local file | |
harness_scaffold |
||
| 6,000 | ||
| vendor, not a local file | ||
mcp_a , mcp_b |
||
| 4,000 each | local files ( | |
session_id
, timestamp
Work per run is a lognormal draw, median 18,000 bytes, sigma 0.8, multiplied by a daily job-mix factor (lognormal, sigma 0.15). So a typical run is 86,033 bytes and the floor is 71.0 percent of it. Every world shares one noise realization and one anchor: only the floor trajectory differs. The dashboard is never handed a harder draw than the anchor gets.
The six worlds:
| world | what happens |
|---|---|
control |
|
| nothing changes | |
creep |
|
| one policy section appended to CLAUDE.md every day, a 3rd MCP server on day 14, a 4th on day 33, 12 tool schemas on day 40 | |
slowcreep |
|
| one policy section every third day, one MCP server on day 33 | |
spike |
|
| someone pastes a 40 KB architecture doc into CLAUDE.md on day 30 | |
harness |
|
| no local file changes at all. The vendor bumps its scaffold on days 12, 31 and 48 | |
workload |
|
| the floor never moves. Work per run doubles on day 30 |
Each policy section is uniquely generated, 380 to 620 bytes, with its own heading and body. I mention that because my last cost article was dropped in review for a fixture that repeated one paragraph thirty times, which was a fair hit.
Growth rates that come out of that: creep
runs at +1.044 percent/day and reaches 112,700 bytes. slowcreep
runs at +0.347 percent/day and reaches 74,885 bytes, up 22.6 percent. Nobody would notice either one by looking.
Every rolling detector gets its own best deal: the tool walks a threshold ladder from 1.05 upward and picks the lowest threshold that raises zero false alarms on the flat world. That is the most generous calibration you can honestly give an alerting rule, because the first thing that happens to a noisy rule in real life is that somebody raises the threshold until it shuts up.
Here is what came back, condensed from the real output:
| world | rolling median w=7 | rolling mean w=7 | EWMA a=0.3 | rolling median w=28 | frozen anchor, tol 2% |
|---|---|---|---|---|---|
control |
|||||
| 0 alarms (T=1.20) | 0 (T=1.20) | 0 (T=1.20) | 0 (T=1.15) | PASS, all 60 days | |
creep (1.04%/d) |
|||||
| 1, on day 40 | |||||
| 1, day 40 | 1, day 40 | 15, from day 30 | BLOCK day 3 | ||
slowcreep (0.35%/d) |
|||||
| 0 | |||||
| 0 | |||||
| 0 | |||||
| 0 | |||||
| BLOCK day 9 | |||||
harness |
|||||
| 0 | |||||
| 0 | |||||
| 0 | |||||
| 1, day 58 | BLOCK day 12 | ||||
spike |
|||||
| 4, day 30 | 2, day 30 | 2, day 30 | 15, day 30 | BLOCK day 30 | |
workload |
|||||
| 4, day 30 | 2, day 30 | 2, day 30 | 14, day 30 | PASS, all 60 days |
Look at the creep
row for the short windows. One alarm each, and it lands on day 40, the day twelve tool schemas were installed. Not on the creep. On the step. The detector is doing exactly what it is built to do, which is notice a discontinuity, and the 51 KB the floor gained by walking there is invisible to it.
Now the long window, w=28
at T=1.15
. The arithmetic said it is blind below 1.00 percent/day. creep
runs at 1.044 percent/day, just above the line, and the detector fires 15 times starting on day 30. slowcreep
runs at 0.347 percent/day, below the line, and it fires zero times in 60 days. The formula predicted both the hit and the miss before the data existed. That is the whole argument in two rows, and it is why I put the arithmetic first.
The long window is not free, by the way. It paid for that sensitivity with 15 alarms on spike
and 14 on workload
.
The harness
world is the one I would put in front of a skeptic.
Nothing in the repository changes. Not one byte. The demo runs diff -rq
between a day-0 snapshot and a day-59 snapshot of every local file that feeds the request:
diff -rq repo_day000 repo_day059: no differences. git diff would print nothing.
sha256 of every local file, day 0 vs day 59:
d0 063d5a8a0fe8267fc4093041701996f728662aab22b0763dd0225de767bff895 ./CLAUDE.md
d0 e3b98852646c31827f303b3762410a3ca42f3a51c2a2201614e21b76fae51e70 ./mcp/a.json
d0 07acba3d89f17bc9467bcbac3f2bcc72cfff4c07ad65694d86691a66a080f80c ./mcp/b.json
d0 25cb751f2526dafd5e0f7273fe29a7abac797424f442cfff28ea82ca013bf8cf ./system_prompt.txt
d0 3dc0f40d1489590ef5a98c464c11378f97e4133aa1a6fcc984fa5d386525440c ./tools.json
d59 063d5a8a0fe8267fc4093041701996f728662aab22b0763dd0225de767bff895 ./CLAUDE.md
d59 e3b98852646c31827f303b3762410a3ca42f3a51c2a2201614e21b76fae51e70 ./mcp/a.json
d59 07acba3d89f17bc9467bcbac3f2bcc72cfff4c07ad65694d86691a66a080f80c ./mcp/b.json
d59 25cb751f2526dafd5e0f7273fe29a7abac797424f442cfff28ea82ca013bf8cf ./system_prompt.txt
d59 3dc0f40d1489590ef5a98c464c11378f97e4133aa1a6fcc984fa5d386525440c ./tools.json
Meanwhile the vendor added 2,600 bytes of tool-call protocol preamble to every request on day 12, another 3,100 on day 31, another 3,400 on day 48. By day 59 the rendered request carries 14.9 percent more floor than it did at pin time, on every single call, forever.
Three instruments looked at that world. git diff
reported nothing. The rolling detectors on the usage log reported nothing (the long window eventually woke up on day 58, which is 46 days late and only after the third bump). The anchor reported this, on day 12, from the same command:
$ drift_anchor_gate.py check worlds/anchor.json worlds/harness/canary/day012.json
anchor 61060 B pinned day 0 tolerance 2.0%
today 63660 B day 12 drift +4.26%
harness_scaffold +2600 B vendor
unchanged (same sha256): instructions, mcp_a, mcp_b, session_id, system_prompt, timestamp, tools_json
from local files: +0 B from the vendor: +2600 B
verdict: BLOCK -> exit 1 (drift +4.26% is above the 2.0% tolerance)
exit=1
The same command, the same anchor, the same day, in the control world:
$ drift_anchor_gate.py check worlds/anchor.json worlds/control/canary/day012.json
today 61060 B day 12 drift +0.00%
verdict: PASS -> exit 0 (drift +0.00% is within the 2.0% tolerance)
exit=0
That line from local files: +0 B from the vendor: +2600 B
is the reason the anchor measures the rendered request and not the files you remembered to list. Your repo is not the thing you pay for. The rendered request is the thing you pay for.
I would not trust this article if it did not have this section, so here it is.
The workload
world: the floor never changes, and on day 30 the work per run doubles (retrieval starts returning twice the chunks, users paste bigger inputs, whatever). The rolling detector fires on day 30 and it is completely correct to do so. The anchor passes all 60 days, because the frozen canary task is not doing more work. Its input really did not drift. The anchor is blind to the entire class of cost growth that lives in the workload (the context tax, where every step re-bills the whole transcript), and that class is real and often bigger than the floor.
Now put spike
and workload
side by side in that results table. Look at the rolling columns. Same detector, same day 30, near-identical alarm counts (4, 2, 2 on the short windows, 15 vs 14 on the long one). From inside the usage log those two worlds are indistinguishable. One of them is a 40 KB doc pasted into an instruction file that you will now pay for on every call until someone deletes it. The other is your product doing more work, which is what you wanted. The dashboard raises the identical alarm for both.
The anchor tells them apart: BLOCK on the paste, PASS on the workload. That is not the anchor being more sensitive. It is the anchor answering a different, narrower question, and answering it cleanly.
So the honest summary is a division of labour, not a replacement. Keep the rolling detector. It catches the paste, the step, and the workload change. Add the anchor. It catches the year.
There is one parameter that can flip the whole result, so I swept it and I am publishing the sweep: how much of a typical request is floor?
The noise in a fleet average comes from the work, not from the floor. The floor is the same on every call. So the less work your runs carry, the quieter your fleet average is, the lower a threshold you can run without false alarms, and the closer your dashboard gets to being a canary all by itself. Here is the sweep on the slowcreep
world, with the w=7
detector recalibrated to its own zero-false-alarm threshold at each work level:
work_median floor share w=7 clean T blind below alarms on slowcreep first
wm002000 95.8% T=1.05 1.40%/day 4 <- sees the creep 33
wm005000 89.8% T=1.05 1.40%/day 6 <- sees the creep 12
wm010000 81.8% T=1.15 4.07%/day 0 -
wm018000 71.7% T=1.15 4.07%/day 0 -
wm030000 59.2% T=2.00 21.90%/day 0 -
wm060000 41.7% T=2.00 21.90%/day 0 -
At a floor share around 90 percent and above, the dashboard sees the creep and this article's headline does not apply to you. The flip in my run happens between 81.8 percent (blind) and 89.8 percent (sees). If your agent's requests are almost entirely system prompt and tool schemas with a tiny user turn on top, you already own a canary. You did not have to freeze anything.
For everyone else, notice what happens at the bottom of that table. At a 42 percent floor share, the ladder had to climb to T=2.00
to stay quiet on a flat fleet. A threshold of 2.00 means "wake me when input cost doubles day over day", and its blind band is 21.9 percent a day. Nobody ships that alert. That is the mechanism in one line: the noisier your work, the higher your threshold must go, and the wider the door you leave open for drift.
I swept the day-to-day noise too, sigma from 0.05 to 0.25. At sigma 0.05, a very quiet fleet, the w=7
detector does catch one alarm on slowcreep
, on day 36 (27 days after the anchor). At every other noise level: zero.
The tool counts bytes. That is deliberate, and it is also the answer to the first objection I would raise if I were reading this.
Drift is a ratio, and a ratio survives any linear rescale: (a*x2) / (a*x1) = x2 / x1
. Whatever your bytes-per-token constant is, whatever your price per token is, it cancels. So I do not need a tokenizer to say the floor grew 22.6 percent, and I refuse to multiply by a made-up constant to make the number look like money.
Caching is the interesting version of this objection. A stable prefix is cacheable, and a cache read can cost about a tenth of a fresh write, so caching changes the level of what you pay by a large factor. It does not change the ratio, so it does not change the verdict. The floor still grew 22.6 percent, you are still paying the drift, and you are paying it on every call. There is a nastier detail underneath: editing an instruction file invalidates the prefix, so on the day someone appends a policy section you pay a full cache write on top (the cache-break detector is about exactly that failure). Slow creep is not one cheap event. It is a cheap event that also throws away your cache, repeatedly.
If you want dollars, multiply. Just do not let the multiplication happen before the comparison.
Two of my own posts are close enough to this one that I owe you a direct answer.
Sliding-Window Spend Guard also uses a window. The difference is where the threshold comes from: there, the cap is absolute ("$X per window, refuse the next call"), and the window is only a way to aggregate. Here, the whole subject is a threshold derived from your own history, which is the thing that fails. An absolute cap is immune to this failure mode by construction. It is also the reason the window guard is a good design and a fleet-relative alert is a trap: one of them has a baseline that cannot be moved by the drift it is supposed to catch.
MCP Tool Pin Verify also pins a hash. The difference is what the pin is for: there it protects the semantic integrity of a tool description against a malicious rug-pull. Here the pin protects a level, and the adversary is not an attacker, it is your own team, being helpful, one policy section at a time. Same mechanism, different failure. Pinning tool manifests does not tell you your floor grew, and this gate does not tell you a tool description turned hostile. Run both.
Both belong to the same family: a pre-execution gate that decides before the money is spent instead of a chart that explains the money after.
A cost gate has exactly one bug it must never have: reporting "all clear" because the artifact it was supposed to measure went missing. My previous gate had that bug (a zero-byte baseline made a division produce 0.0
, which was below the threshold, which was a pass, on 2.25 million tokens). It was caught in review and the article was dropped. Fair.
So this one has three exits and a rule:
0 PASS
drift within tolerance, canary frozen, every declared field present and non-empty.1 BLOCK
drift above tolerance.2 STRUCTURAL
the input is unusable.There is no path where a missing, emptied or truncated artifact returns 0. (A record that faithfully reports a genuinely smaller request is a PASS — that is the whole point of measuring level, not a fail-open. What cannot happen is the measurement vanishing, or a field dropping out, and being scored as "no drift.") Nine degenerate inputs, all from the real run:
--- dropped_field ---
STRUCTURAL: declared field missing from today's record: tools_json
STRUCTURAL: a bytes-only check would have called this -42.6% drift and passed it
verdict: STRUCTURAL -> exit 2 (fail-closed: unusable input is never a pass)
exit=2
--- zero_byte_field ---
STRUCTURAL: declared field rendered zero bytes: mcp_b
exit=2
--- unknown_field ---
STRUCTURAL: undeclared field in today's record: mcp_c
STRUCTURAL: you do not know everything your harness is sending; re-pin deliberately
exit=2
--- task_changed ---
STRUCTURAL: canary task changed: task_sha256 c81db92e6a9f... != pinned 7439b8b0addf...
STRUCTURAL: a canary that drifts is not a canary
exit=2
--- volatile_grew ---
STRUCTURAL: volatile field session_id changed length: 36 B pinned, 96 B today
STRUCTURAL: a volatile field with a moving length is not volatile, it is drift in disguise
exit=2
Plus missing
, empty
, truncated
, and inconsistent_total
(the parts do not sum to the stated total). Nine cases, nine exit 2s. The dropped_field
line is the one to stare at: a naive bytes-only comparison reads that record as 42.6 percent cheaper than the anchor and passes it. That is the fail-open shape, and it is why the manifest is a manifest and not a number.
And on the typical scenario, the one this tool exists for, it fails closed in the direction that costs someone a conversation: slowcreep
day 8 passes at +1.80 percent, day 9 blocks at +2.79 percent, exit 1, attribution instructions +1706 B local:CLAUDE.md
. Your CI goes red because somebody added four paragraphs of rules over nine days. That is the intended behaviour, and if it annoys you, re-pin the anchor. Deliberately. In a commit. With your name on it.
It is 1,726 bytes and there is nothing clever in it:
{
"canary_id": "floor-canary-v1",
"pinned_at": "day 0",
"tolerance_pct": 2.0,
"task_sha256": "7439b8b0addf5009c942bb2f1c42c0c4e3489b13990957f8a47253aaa429a8c8",
"rendered_bytes": 61060,
"manifest": {
"tools_json": {"bytes": 26000, "sha256": "3dc0f40d...", "origin": "local:tools.json", "volatile": false},
"system_prompt": {"bytes": 12000, "sha256": "25cb751f...", "origin": "local:system_prompt.txt","volatile": false},
"instructions": {"bytes": 9000, "sha256": "063d5a8a...", "origin": "local:CLAUDE.md", "volatile": false},
"harness_scaffold": {"bytes": 6000, "sha256": "39b235eb...", "origin": "vendor", "volatile": false},
"mcp_a": {"bytes": 4000, "sha256": "e3b98852...", "origin": "local:mcp/a.json", "volatile": false},
"mcp_b": {"bytes": 4000, "sha256": "07acba3d...", "origin": "local:mcp/b.json", "volatile": false},
"session_id": {"bytes": 36, "sha256": "96311dd8...", "origin": "runtime", "volatile": true},
"timestamp": {"bytes": 24, "sha256": "2956a7fc...", "origin": "runtime", "volatile": true}
}
}
(Hashes truncated here for width. They are full-length in the file.) One detail worth pausing on: instructions
is pinned at 063d5a8a...
, and that is the same digest shasum
printed for CLAUDE.md
on day 59 of the harness world. The file is provably untouched, and the request still grew.
Today's record is the same shape: per-field byte counts and digests for the rendered request. Byte counts and hashes, not payloads, which is the shape a privacy-conscious request log already has. If your harness can dump the request it is about to send, you can produce this. If it cannot, that is a finding.
Three files. All of it, because a reproducibility claim you cannot check is just a claim.
drift_anchor_gate.py
:
#!/usr/bin/env python3
"""drift_anchor_gate.py - offline, keyless, zero-network, read-only, stdlib-only (Python 3.13).
One question: has the INPUT COST of a known-good run gone up since a known-good day?
A rolling baseline (median / mean / EWMA over your fleet) answers that question by
comparing today against your own recent history. It is a detector of SPEED. The
invoice is charged on LEVEL. This tool answers the same question against a baseline
that does not move: an anchor pinned on day 0 to a FROZEN canary task, so the only
thing that can change between then and now is what your harness wraps around it.
check one day against the anchor. Exit 0 PASS / 1 BLOCK / 2 STRUCTURAL.
replay N days, anchor and rolling detectors side by side on the same log.
sweep the two parameters that could flip the result: floor share, day noise.
blindband pure arithmetic, no data: the growth rate a rolling window cannot see.
Exit contract (fail-closed):
0 PASS drift within tolerance, canary frozen, every declared field present and non-empty
1 BLOCK drift above tolerance
2 STRUCTURAL input unusable: unreadable, empty, truncated, a declared field missing or
rendered zero bytes, an undeclared field present, the canary task changed,
a volatile field that changed length, a record whose parts do not sum to its total.
There is no path where a missing, emptied or truncated artifact returns 0, and a
dropped or zeroed field fails closed too. A gate that answers "no drift detected"
because the artifact vanished is worse than no gate.
This tool never writes, never opens a socket, never imports anything outside the stdlib.
"""
import hashlib
import json
import os
import statistics
import sys
LADDER = [1.05, 1.10, 1.15, 1.20, 1.25, 1.30, 1.40, 1.50, 2.00]
FAMILIES = [("rolling median w=7", "median", 7),
("rolling mean w=7", "mean", 7),
("EWMA alpha=0.3", "ewma", 7),
("rolling median w=28", "median", 28)]
ALPHA = 0.3
def load_json(path, what):
if not os.path.exists(path):
die("%s not found: %s" % (what, path))
if os.path.getsize(path) == 0:
die("%s is zero bytes: %s" % (what, path))
try:
with open(path) as fh:
return json.load(fh)
except (ValueError, OSError) as exc:
die("%s is not readable JSON: %s (%s)" % (what, path, exc.__class__.__name__))
def die(msg):
print("STRUCTURAL: %s" % msg)
print("verdict: STRUCTURAL -> exit 2 (fail-closed: unusable input is never a pass)")
sys.exit(2)
def check(anchor, today, quiet=False):
"""Return (verdict, drift_pct, lines). verdict in PASS / BLOCK / STRUCTURAL."""
out = []
man = anchor["manifest"]
fields = today.get("fields")
if not isinstance(fields, dict) or not fields:
return "STRUCTURAL", 0.0, ["no fields in today's record"]
if today.get("canary_id") != anchor["canary_id"]:
return "STRUCTURAL", 0.0, ["canary_id mismatch: %r vs pinned %r"
% (today.get("canary_id"), anchor["canary_id"])]
if today.get("task_sha256") != anchor["task_sha256"]:
return "STRUCTURAL", 0.0, ["canary task changed: task_sha256 %s... != pinned %s..."
% (str(today.get("task_sha256"))[:12], anchor["task_sha256"][:12]),
"a canary that drifts is not a canary"]
missing = [k for k in man if k not in fields]
unknown = [k for k in fields if k not in man]
empty = [k for k, v in fields.items() if v.get("bytes", 0) <= 0]
if missing:
naive = sum(v["bytes"] for v in fields.values()) / anchor["rendered_bytes"] - 1
return "STRUCTURAL", 0.0, [
"declared field missing from today's record: %s" % ", ".join(sorted(missing)),
"a bytes-only check would have called this %+.1f%% drift and passed it" % (naive * 100)]
if unknown:
return "STRUCTURAL", 0.0, [
"undeclared field in today's record: %s" % ", ".join(sorted(unknown)),
"you do not know everything your harness is sending; re-pin deliberately"]
if empty:
return "STRUCTURAL", 0.0, ["declared field rendered zero bytes: %s" % ", ".join(sorted(empty))]
for k, v in man.items():
if v.get("volatile") and fields[k]["bytes"] != v["bytes"]:
return "STRUCTURAL", 0.0, [
"volatile field %s changed length: %d B pinned, %d B today"
% (k, v["bytes"], fields[k]["bytes"]),
"a volatile field with a moving length is not volatile, it is drift in disguise"]
total = sum(v["bytes"] for v in fields.values())
if total != today.get("rendered_bytes"):
return "STRUCTURAL", 0.0, ["record does not sum: fields %d B, rendered_bytes %d B"
% (total, today.get("rendered_bytes"))]
drift = total / anchor["rendered_bytes"] - 1
tol = anchor["tolerance_pct"] / 100.0
deltas = sorted(((fields[k]["bytes"] - man[k]["bytes"], k) for k in man),
key=lambda t: -abs(t[0]))
local = sum(d for d, k in deltas if man[k]["origin"].startswith("local"))
vendor = sum(d for d, k in deltas if not man[k]["origin"].startswith("local"))
if not quiet:
out.append("anchor %8d B pinned %s tolerance %.1f%%"
% (anchor["rendered_bytes"], anchor["pinned_at"], anchor["tolerance_pct"]))
out.append("today %8d B day %-3s drift %+.2f%%"
% (total, today.get("day", "?"), drift * 100))
out.append("")
for d, k in deltas:
if d:
out.append(" %-18s %+8d B %s" % (k, d, man[k]["origin"]))
same = [k for d, k in deltas if d == 0 and fields[k]["sha256"] == man[k]["sha256"]]
if same:
out.append(" unchanged (same sha256): %s" % ", ".join(sorted(same)))
out.append("")
out.append(" from local files: %+d B from the vendor: %+d B" % (local, vendor))
verdict = "BLOCK" if drift > tol else "PASS"
return verdict, drift * 100, out
def cmd_check(args):
anchor = load_json(args[0], "anchor")
today = load_json(args[1], "today's record")
verdict, drift, lines = check(anchor, today)
if verdict == "STRUCTURAL":
for ln in lines:
print("STRUCTURAL: %s" % ln)
print("verdict: STRUCTURAL -> exit 2 (fail-closed: unusable input is never a pass)")
return 2
body = "\n".join(lines)
print(body)
print("")
if verdict == "BLOCK":
print("verdict: BLOCK -> exit 1 (drift %+.2f%% is above the %.1f%% tolerance)"
% (drift, anchor["tolerance_pct"]))
else:
print("verdict: PASS -> exit 0 (drift %+.2f%% is within the %.1f%% tolerance)"
% (drift, anchor["tolerance_pct"]))
print("report-sha256: %s" % hashlib.sha256(body.encode()).hexdigest())
return 1 if verdict == "BLOCK" else 0
def daily_means(path):
if not os.path.exists(path) or os.path.getsize(path) == 0:
die("usage log missing or empty: %s" % path)
days = {}
with open(path) as fh:
for line in fh:
r = json.loads(line)
days.setdefault(r["day"], []).append(r["input_bytes"])
if not days:
die("usage log has no records: %s" % path)
return [statistics.mean(days[d]) for d in sorted(days)]
def alarms(series, kind, w, t):
"""Days on which today's mean exceeds the baseline built from the PREVIOUS w days by factor t."""
hits = []
ew = None
for i, x in enumerate(series):
if i >= w:
if kind == "median":
base = statistics.median(series[i - w:i])
elif kind == "mean":
base = statistics.mean(series[i - w:i])
else:
base = ew
if base and x / base > t:
hits.append(i)
if kind == "ewma":
ew = x if ew is None else ALPHA * x + (1 - ALPHA) * ew
return hits
def calibrate(control, kind, w):
"""The lowest threshold on the ladder that raises zero false alarms on a flat world."""
for t in LADDER:
if not alarms(control, kind, w, t):
return t
return None
def blind_band(t, w):
"""Uniform daily growth g is invisible while (1+g)^(w/2) <= T. Arithmetic, not a finding."""
return t ** (2.0 / w) - 1.0
def cmd_blindband(_args):
print("blind band of a rolling window: it compares today against a baseline centred")
print("about w/2 days back, so a uniform daily growth g only trips it when (1+g)^(w/2) > T.")
print("solve for g: g* = T^(2/w) - 1 any growth slower than g* is invisible forever.")
print("")
print(" window threshold invisible below compounds to, over 30 days")
for w, t in ((7, 1.15), (7, 1.20), (14, 1.20), (28, 1.20), (28, 1.10)):
g = blind_band(t, w)
print(" w=%-4d T=%.2f %6.2f%%/day x%.2f" % (w, t, g * 100, (1 + g) ** 30))
print("")
print("this is arithmetic. it is true before you collect a single byte of data.")
print("the measurement is where real context creep lands relative to that band.")
return 0
def cmd_replay(args):
world, ctrl = args[0], args[1]
anchor = load_json(os.path.join(os.path.dirname(world), "anchor.json"), "anchor")
events = load_json(os.path.join(world, "events.json"), "events")
series = daily_means(os.path.join(world, "usage.jsonl"))
control = daily_means(os.path.join(ctrl, "usage.jsonl"))
print("world: %s days: %d runs/day: fixed" % (os.path.basename(world), len(series)))
floor0 = anchor["rendered_bytes"]
typical = statistics.mean(control) # a flat fleet, so this is the work profile, not the drift
print("floor at pin time %d B typical run %d B floor share %.1f%%"
% (floor0, round(typical), 100.0 * floor0 / typical))
print("")
print(" detector threshold alarms first on the day of")
for name, kind, w in FAMILIES:
t = calibrate(control, kind, w)
if t is None:
print(" %-20s none clean on a flat world" % name)
continue
hits = alarms(series, kind, w, t)
first = hits[0] if hits else None
ev = events.get(str(first), "nothing: this one is about the creep") if first is not None else "-"
print(" %-20s T=%.2f %2d %-5s %s"
% (name, t, len(hits), first if first is not None else "-", ev))
g = blind_band(t, w)
print(" %-20s blind below %.2f%%/day (x%.2f over 30 days)" % ("", g * 100, (1 + g) ** 30))
first_block, first_struct, drift_at, final = None, None, 0.0, floor0
for day in range(len(series)):
rec = load_json(os.path.join(world, "canary", "day%03d.json" % day), "record")
verdict, drift, _ = check(anchor, rec, quiet=True)
if verdict == "BLOCK" and first_block is None:
first_block, drift_at = day, drift
if verdict == "STRUCTURAL" and first_struct is None:
first_struct = day
final = rec["rendered_bytes"]
print("")
if first_block is None and first_struct is None:
print(" frozen anchor tol=%.1f%% PASS on all %d days: the frozen input never drifted"
% (anchor["tolerance_pct"], len(series)))
if first_block is not None:
print(" frozen anchor tol=%.1f%% BLOCK on day %d (drift %+.2f%%) -> exit 1"
% (anchor["tolerance_pct"], first_block, drift_at))
if first_struct is not None:
print(" frozen anchor tol=%.1f%% STRUCTURAL on day %d (%s) -> exit 2"
% (anchor["tolerance_pct"], first_struct,
events.get(str(first_struct), "the set of fields changed")))
print(" floor on the last day %d B (%+.1f%% against the anchor)"
% (final, (final / floor0 - 1) * 100))
return 1 if first_block is not None else (2 if first_struct is not None else 0)
def cmd_sweep(args):
root, label = args[0].rstrip("/"), args[1]
if not os.path.isdir(root):
die("sweep root not found: %s" % root)
anchor = load_json(os.path.join(os.path.dirname(root), "anchor.json"), "anchor")
print(" %-10s floor share w=7 clean T blind below alarms on slowcreep first" % label)
for name in sorted(os.listdir(root)):
d = os.path.join(root, name)
ctrl = daily_means(os.path.join(d, "control", "usage.jsonl"))
slow = daily_means(os.path.join(d, "slowcreep", "usage.jsonl"))
t = calibrate(ctrl, "median", 7)
hits = alarms(slow, "median", 7, t)
share = 100.0 * anchor["rendered_bytes"] / statistics.mean(ctrl)
g = blind_band(t, 7) * 100
print(" %-10s %6.1f%% T=%.2f %6.2f%%/day %2d %-18s %s"
% (name, share, t, g, len(hits),
"<- sees the creep" if hits else "", hits[0] if hits else "-"))
print("")
print(" neither parameter reaches the frozen anchor. It reads a deterministic input, so it")
print(" carries no work and no day-to-day noise: floor share 100%%, tolerance %.1f%%, same"
% anchor["tolerance_pct"])
print(" BLOCK day in every row above. The rows only move the DASHBOARD.")
return 0
USAGE = """usage:
drift_anchor_gate.py check <anchor.json> <today.json>
drift_anchor_gate.py replay <world_dir> <control_dir>
drift_anchor_gate.py sweep <sweep_root> <label>
drift_anchor_gate.py blindband
"""
if __name__ == "__main__":
cmds = {"check": cmd_check, "replay": cmd_replay, "sweep": cmd_sweep, "blindband": cmd_blindband}
if len(sys.argv) < 2 or sys.argv[1] not in cmds:
sys.stdout.write(USAGE)
sys.exit(2)
sys.exit(cmds[sys.argv[1]](sys.argv[2:]))
make_worlds.py
, the fixture builder. Seeded, and the only thing here that writes anything:
#!/usr/bin/env python3
"""make_worlds.py - build the fixtures for drift_anchor_gate.py.
Offline, keyless, zero-network, stdlib-only, seeded. This is the only script here
that writes anything. The gate itself never writes.
Six worlds share ONE anchor (day 0 is byte-identical everywhere) and ONE noise
realization per (work_median, sigma_day) pair. Only the floor trajectory differs
between worlds, so the dashboard is never handed a harder noise draw than the
anchor. Runs per day are FIXED at 40 on purpose: that removes run-count noise,
which is a handicap in the dashboard's favour.
"""
import hashlib
import json
import os
import random
import shutil
HERE = os.path.dirname(os.path.abspath(__file__))
OUT = os.path.join(HERE, "worlds")
DAYS = 60
RUNS_PER_DAY = 40
BASE_SEED = 20260715
WORK_MEDIAN = 18_000 # bytes of real per-run work: the user turn, retrieved chunks, history
WORK_SIGMA = 0.8 # lognormal spread of work across runs
SIGMA_DAY = 0.15 # lognormal spread of the daily job mix
TOL_PCT = 2.0
TASK = ("canary task: read the pinned fixture file, return its word count as JSON. "
"do not call tools. do not browse. answer in one line.")
FLOOR0 = {
"harness_scaffold": 6_000, # vendor-injected: tool-call protocol preamble + safety wrapper
"system_prompt": 12_000,
"tools_json": 26_000,
"instructions": 9_000, # CLAUDE.md / AGENTS.md
"mcp_a": 4_000,
"mcp_b": 4_000,
}
VOLATILE = {"session_id": 36, "timestamp": 24}
ORIGIN = {
"harness_scaffold": "vendor",
"system_prompt": "local:system_prompt.txt",
"tools_json": "local:tools.json",
"instructions": "local:CLAUDE.md",
"mcp_a": "local:mcp/a.json",
"mcp_b": "local:mcp/b.json",
"mcp_c": "local:mcp/c.json",
"mcp_d": "local:mcp/d.json",
"session_id": "runtime",
"timestamp": "runtime",
}
WORDS = ("agent budget cache canary context deploy drift enforce escalate fixture gate harness "
"ingest invariant ledger manifest observe payload pin policy prompt quota render replay "
"retry rollback runbook schema scope session snapshot spend threshold token trace vendor "
"window workspace approve reject timeout latency shard cursor batch").split()
SUBJECTS = ["retry budget", "tool allowlist", "pii redaction", "escalation path", "commit policy",
"sandbox scope", "timeout ladder", "cache warmup", "rollback drill", "on-call handoff",
"schema review", "cost ceiling", "log retention", "vendor swap", "eval cadence",
"prompt review", "secret handling", "shell allowlist", "diff size cap", "branch policy"]
def text_of(rng, n, tag):
"""Deterministic ASCII text of exactly n bytes."""
if n <= 0:
return ""
parts = [tag, ":"]
size = len(tag) + 1
while size < n:
w = rng.choice(WORDS)
parts.append(" " + w)
size += len(w) + 1
s = "".join(parts)[:n]
return s + " " * (n - len(s))
def rule_section(rng, idx):
"""One appended policy section. Unique subject, unique body, 380..620 bytes."""
n = rng.randint(380, 620)
head = "\n\n## Rule %03d: %s\n" % (idx, SUBJECTS[idx % len(SUBJECTS)])
return head + text_of(rng, n - len(head), "rationale-%03d" % idx)
_c = random.Random(BASE_SEED ^ 0xF100)
BASE = {k: text_of(_c, v, k) for k, v in FLOOR0.items()}
_s = random.Random(BASE_SEED ^ 0x5EC7)
SECTIONS = [rule_section(_s, i) for i in range(70)]
EXTRA_TOOLS = "".join(text_of(_c, 1_150, "tool-schema-%02d" % i) for i in range(12)) # 13,800 B
MCP_C = text_of(_c, 4_200, "mcp_c")
MCP_D = text_of(_c, 3_800, "mcp_d")
SPIKE_DOC = text_of(_c, 40_000, "pasted-architecture-doc")
BUMP = {12: text_of(_c, 2_600, "scaffold-bump-1"),
31: text_of(_c, 3_100, "scaffold-bump-2"),
48: text_of(_c, 3_400, "scaffold-bump-3")}
EVENTS = {
"control": {},
"creep": {14: "mcp: +3rd server", 33: "mcp: +4th server", 40: "tools: +12 schemas"},
"slowcreep": {33: "mcp: +3rd server"},
"spike": {30: "instructions: +40 KB doc pasted"},
"harness": {12: "vendor: scaffold bump 1", 31: "vendor: scaffold bump 2", 48: "vendor: scaffold bump 3"},
"workload": {30: "work per run doubles (floor untouched)"},
}
WORLDS = list(EVENTS)
def render_floor(world, day):
"""Field -> content string for this world on this day. Day 0 is identical in every world."""
f = dict(BASE)
if world == "creep":
n = day # one policy section per day
elif world == "slowcreep":
n = day // 3 # one policy section every third day
else:
n = 0
if n:
f["instructions"] = BASE["instructions"] + "".join(SECTIONS[:n])
if world == "creep":
if day >= 40:
f["tools_json"] = BASE["tools_json"] + EXTRA_TOOLS
if day >= 14:
f["mcp_c"] = MCP_C
if day >= 33:
f["mcp_d"] = MCP_D
if world == "slowcreep" and day >= 33:
f["mcp_c"] = MCP_C
if world == "spike" and day >= 30:
f["instructions"] = f["instructions"] + SPIKE_DOC
if world == "harness":
add = "".join(v for d, v in sorted(BUMP.items()) if day >= d)
f["harness_scaffold"] = BASE["harness_scaffold"] + add
return f
def work_scale(world, day):
"""The only world that changes the WORK per run rather than the floor."""
return 2.0 if (world == "workload" and day >= 30) else 1.0
def sha(s):
return hashlib.sha256(s.encode("utf-8")).hexdigest()
def dump_fields(fields):
"""Rendered-request record: per-field byte count + sha256. The shape of a privacy-safe request log."""
rec = {k: {"bytes": len(v), "sha256": sha(v)} for k, v in fields.items()}
for k, n in VOLATILE.items():
rec[k] = {"bytes": n, "sha256": sha(k * n)}
return rec
def total_bytes(fields):
return sum(len(v) for v in fields.values()) + sum(VOLATILE.values())
def noise_matrix(work_median, sigma_day, seed):
"""Per-run work bytes. Identical across worlds for a given (work_median, sigma_day)."""
rng = random.Random(seed)
rows = []
for _ in range(DAYS):
dm = rng.lognormvariate(0.0, sigma_day)
rows.append([max(1, int(work_median * rng.lognormvariate(0.0, WORK_SIGMA) * dm))
for _ in range(RUNS_PER_DAY)])
return rows
def write_usage(path, world, noise):
"""usage.jsonl: one line per run, the log a fleet dashboard already has."""
with open(path, "w") as fh:
for day in range(DAYS):
floor = total_bytes(render_floor(world, day))
k = work_scale(world, day)
for run in range(RUNS_PER_DAY):
fh.write(json.dumps({"day": day, "run": run,
"input_bytes": floor + int(noise[day][run] * k)}) + "\n")
def main():
if os.path.isdir(OUT):
shutil.rmtree(OUT)
os.makedirs(OUT)
day0 = render_floor("control", 0)
anchor = {
"canary_id": "floor-canary-v1",
"pinned_at": "day 0",
"tolerance_pct": TOL_PCT,
"task_sha256": sha(TASK),
"rendered_bytes": total_bytes(day0),
"manifest": {},
}
for k, rec in dump_fields(day0).items():
anchor["manifest"][k] = {"bytes": rec["bytes"], "sha256": rec["sha256"],
"origin": ORIGIN[k], "volatile": k in VOLATILE}
with open(os.path.join(OUT, "anchor.json"), "w") as fh:
json.dump(anchor, fh, indent=2, sort_keys=True)
noise = noise_matrix(WORK_MEDIAN, SIGMA_DAY, BASE_SEED)
for world in WORLDS:
wd = os.path.join(OUT, world)
os.makedirs(os.path.join(wd, "canary"))
for day in range(DAYS):
fields = render_floor(world, day)
rec = {"canary_id": "floor-canary-v1", "day": day, "task_sha256": sha(TASK),
"rendered_bytes": total_bytes(fields), "fields": dump_fields(fields)}
with open(os.path.join(wd, "canary", "day%03d.json" % day), "w") as fh:
json.dump(rec, fh, indent=2, sort_keys=True)
write_usage(os.path.join(wd, "usage.jsonl"), world, noise)
with open(os.path.join(wd, "events.json"), "w") as fh:
json.dump({str(k): v for k, v in EVENTS[world].items()}, fh, indent=2, sort_keys=True)
for day in (0, 59):
d = os.path.join(OUT, "harness", "repo_day%03d" % day)
os.makedirs(os.path.join(d, "mcp"))
f = render_floor("harness", day)
open(os.path.join(d, "system_prompt.txt"), "w").write(f["system_prompt"])
open(os.path.join(d, "CLAUDE.md"), "w").write(f["instructions"])
open(os.path.join(d, "tools.json"), "w").write(f["tools_json"])
open(os.path.join(d, "mcp", "a.json"), "w").write(f["mcp_a"])
open(os.path.join(d, "mcp", "b.json"), "w").write(f["mcp_b"])
dg = os.path.join(OUT, "degenerate")
os.makedirs(dg)
base = json.load(open(os.path.join(OUT, "control", "canary", "day000.json")))
def variant(name, mutate):
r = json.loads(json.dumps(base))
mutate(r)
with open(os.path.join(dg, name + ".json"), "w") as fh:
json.dump(r, fh, indent=2, sort_keys=True)
def drop_tools(r):
r["fields"].pop("tools_json")
r["rendered_bytes"] = sum(v["bytes"] for v in r["fields"].values())
def zero_mcp(r):
r["fields"]["mcp_b"] = {"bytes": 0, "sha256": sha("")}
r["rendered_bytes"] = sum(v["bytes"] for v in r["fields"].values())
def unknown_field(r):
r["fields"]["mcp_c"] = {"bytes": 4_200, "sha256": sha(MCP_C)}
r["rendered_bytes"] = sum(v["bytes"] for v in r["fields"].values())
def task_changed(r):
r["task_sha256"] = sha(TASK + " and also list the files you can see")
def volatile_grew(r):
r["fields"]["session_id"] = {"bytes": 96, "sha256": sha("x" * 96)}
r["rendered_bytes"] = sum(v["bytes"] for v in r["fields"].values())
def bad_sum(r):
r["rendered_bytes"] = 61_060 - 20_000
variant("dropped_field", drop_tools)
variant("zero_byte_field", zero_mcp)
variant("unknown_field", unknown_field)
variant("task_changed", task_changed)
variant("volatile_grew", volatile_grew)
variant("inconsistent_total", bad_sum)
open(os.path.join(dg, "empty.json"), "w").close()
open(os.path.join(dg, "truncated.json"), "w").write('{"canary_id": "floor-canary-v1", "day": 0,')
for wm in (2_000, 5_000, 10_000, 18_000, 30_000, 60_000):
seed = BASE_SEED + wm + 15 * 1_000_003
nz = noise_matrix(wm, SIGMA_DAY, seed)
for world in ("control", "slowcreep"):
d = os.path.join(OUT, "sweep_floorshare", "wm%06d" % wm, world)
os.makedirs(d)
write_usage(os.path.join(d, "usage.jsonl"), world, nz)
shutil.copy(os.path.join(OUT, world, "events.json"), os.path.join(d, "events.json"))
for sig in (0.05, 0.10, 0.15, 0.20, 0.25):
seed = BASE_SEED + WORK_MEDIAN + int(sig * 100) * 1_000_003
nz = noise_matrix(WORK_MEDIAN, sig, seed)
for world in ("control", "slowcreep"):
d = os.path.join(OUT, "sweep_sigma", "s%03d" % int(sig * 100), world)
os.makedirs(d)
write_usage(os.path.join(d, "usage.jsonl"), world, nz)
shutil.copy(os.path.join(OUT, world, "events.json"), os.path.join(d, "events.json"))
print("worlds built: %s" % ", ".join(WORLDS))
print("anchor rendered_bytes: %d B tolerance: %.1f%%" % (anchor["rendered_bytes"], TOL_PCT))
for world in WORLDS:
a, b = total_bytes(render_floor(world, 0)), total_bytes(render_floor(world, 59))
rate = ((b / a) ** (1 / 59) - 1) * 100 if b != a else 0.0
print(" %-10s floor day0 %7d B -> day59 %7d B (x%.2f, %+.3f%%/day)"
% (world, a, b, b / a, rate))
if __name__ == "__main__":
main()
run_demo.sh
:
#!/usr/bin/env bash
set -u
cd "$(dirname "$0")"
hr() { printf '\n%s\n%s\n' "$1" "$(printf '=%.0s' $(seq 1 ${#1}))"; }
hr "0. THE PREDICTION (arithmetic, before any data exists)"
python3 drift_anchor_gate.py blindband
hr "1. BUILD THE WORLDS (seeded, deterministic, no network)"
python3 make_worlds.py
hr "2. KILLER DEMO: the harness world. Every local file is byte-identical to day 0."
diff -rq worlds/harness/repo_day000 worlds/harness/repo_day059 \
&& echo "diff -rq repo_day000 repo_day059: no differences. git diff would print nothing."
echo ""
echo "sha256 of every local file, day 0 vs day 59:"
( cd worlds/harness/repo_day000 && find . -type f | sort | xargs shasum -a 256 ) | sed 's/^/ d0 /'
( cd worlds/harness/repo_day059 && find . -type f | sort | xargs shasum -a 256 ) | sed 's/^/ d59 /'
echo ""
echo "--- same anchor, same day 12, two worlds ---"
echo ""
echo "\$ drift_anchor_gate.py check worlds/anchor.json worlds/control/canary/day012.json"
python3 drift_anchor_gate.py check worlds/anchor.json worlds/control/canary/day012.json
echo "exit=$?"
echo ""
echo "\$ drift_anchor_gate.py check worlds/anchor.json worlds/harness/canary/day012.json"
python3 drift_anchor_gate.py check worlds/anchor.json worlds/harness/canary/day012.json
echo "exit=$?"
hr "3. SIX WORLDS, ONE ANCHOR, ONE LOG. Each rolling detector gets its own best threshold."
for w in control creep slowcreep harness spike workload; do
echo ""
python3 drift_anchor_gate.py replay "worlds/$w" worlds/control
echo " replay exit=$?"
done
hr "4. THE TYPICAL SCENARIO, AT THE BOUNDARY. The gate must fail CLOSED on slow creep."
echo ""
echo "\$ drift_anchor_gate.py check worlds/anchor.json worlds/slowcreep/canary/day008.json"
python3 drift_anchor_gate.py check worlds/anchor.json worlds/slowcreep/canary/day008.json
echo "exit=$?"
echo ""
echo "\$ drift_anchor_gate.py check worlds/anchor.json worlds/slowcreep/canary/day009.json"
python3 drift_anchor_gate.py check worlds/anchor.json worlds/slowcreep/canary/day009.json
echo "exit=$?"
hr "5. DEGENERATE INPUTS: every one of these must fail closed."
for f in missing empty truncated dropped_field zero_byte_field unknown_field task_changed volatile_grew inconsistent_total; do
echo ""
echo "--- $f ---"
python3 drift_anchor_gate.py check worlds/anchor.json "worlds/degenerate/$f.json"
echo "exit=$?"
done
echo ""
echo "--- clean day 0 (the control: it must still pass) ---"
python3 drift_anchor_gate.py check worlds/anchor.json worlds/control/canary/day000.json
echo "exit=$?"
hr "6. SWEEP A: the one parameter that flips the result. How much of the request is floor?"
python3 drift_anchor_gate.py sweep worlds/sweep_floorshare work_median
hr "7. SWEEP B: day-to-day noise. Does the dashboard wake up at any of these?"
python3 drift_anchor_gate.py sweep worlds/sweep_sigma sigma_day
Run it: bash run_demo.sh > output.txt
. It takes a few seconds, touches no network, needs no keys, and produces the same bytes every time. Mine hashes to 1772e695cb75f79d9e3f162ed4c49477a329703610cdf5ddff54cec2cc4da62a
.
usage.jsonl
and your real request dumps.Dump the rendered request for one frozen, deterministic task. Save the byte counts and hashes as an anchor with today's date. Add one CI step that re-renders that same task and compares. Set the tolerance somewhere near 2 percent, because a deterministic input has no noise to hide in, and let it fail the build.
Then keep your dashboard exactly as it is. It is watching for the paste and the traffic step, and it is good at that. It was never watching for the year.
One thing I have not settled, and I would genuinely like a second opinion: the re-pin. Every legitimate change (a new MCP server, a needed rule) forces a re-pin, and a re-pin resets the baseline. Do it a dozen times and you have reinvented a rolling baseline with extra steps, just with human latency in the loop. My current answer is that the re-pin has to carry the cumulative drift since the original pin, so the number you are approving is "the floor is now 47 percent above where it was in March", not "plus 4 percent since last week". I have not built that yet. If you have run a pinned baseline in production for longer than a quarter, I want to know how you kept the pin honest.
I write about pre-execution control for AI agents: gates that decide before the money moves, not charts that explain it afterwards. Every post ships a tool you can actually run, offline, with no keys. Follow along if that is your kind of thing, and if your context floor has a story, put it in the comments. I am especially interested in anyone who has watched a vendor bump a harness without telling them.