The third time I hit send, nothing in the terminal even flinched. Same wrong label. complaint, for a sentence that was begging to be a question: "Where is the rubric?"
I had the failing fixture in retry.json. The bytes were identical to the first attempt. My mood was not. If the canonical request did not change, what new evidence do I think a model is going to hand me?
That was a course lab in Halifax, not a startup demo. One file. One label set. A lot of false confidence.
I was sorting short student comments into question, complaint, or praise. The text was boring on purpose, because boring text makes a sloppy method obvious. I kept a scratch note beside the payload and rewrote the note until I felt the little click of "fixed." Then I resent the old JSON.
The note never entered the request. Have you graded a comment and called it a code change? I had. Four times, if I am honest about the evening.
The goal was a receipt, not a chatbot. Freeze the request. Refuse the send when the diff is empty. Keep the implementation in the standard library so a lab image with no pip can still run it. I wanted the failure louder than my optimism: changed keys on stdout, and an exit code a shell can actually see.
You need Python 3.9 or newer, a writable folder, and no third-party packages. Check the version before you blame the fixture.
python3 --version
I traced the branches by hand. I did not capture this on a remote box, and I will not paste a hash I did not recompute in your terminal. If your run disagrees with the actions below, trust your run and treat my trace as the bug.
hashlib and json are already in the standard library. The contract is dull, which is why I trust it. json.dumps with sort_keys=True gives a stable key order, and SHA-256 is a pure function of those UTF-8 bytes. I kept the Python docs for json.dumps and hashlib.sha256 open beside the script. No model paper required. The concept breaks before any model speaks.
Think of the journal as a coat check, not a diary. You do not get a new ticket because you feel colder. You get a new ticket when the coat is different. Key order is the lining turned inside out. Same coat.
#!/usr/bin/env python3
"""Block a model send when the canonical request did not change."""
import hashlib
import json
import sys
from pathlib import Path
def canonicalize(request):
return json.dumps(
request,
sort_keys=True,
ensure_ascii=False,
separators=(",", ":"),
)
def digest(request):
raw = canonicalize(request).encode("utf-8")
return hashlib.sha256(raw).hexdigest()
def changed_keys(previous, current):
keys = sorted(set(previous) | set(current))
changed = []
for key in keys:
if previous.get(key) != current.get(key):
changed.append(key)
return changed
def decide(journal_path, request):
current_hash = digest(request)
if not journal_path.exists():
payload = {"hash": current_hash, "request": request}
journal_path.write_text(
json.dumps(payload, indent=2) + "\n",
encoding="utf-8",
)
return {
"action": "send",
"reason": "no prior entry",
"hash": current_hash,
"changed": sorted(request),
}
previous = json.loads(journal_path.read_text(encoding="utf-8"))
changed = changed_keys(previous["request"], request)
if not changed:
return {
"action": "block",
"reason": "empty diff",
"hash": current_hash,
"changed": [],
}
payload = {
"hash": current_hash,
"request": request,
"changed_from": previous["hash"],
}
journal_path.write_text(
json.dumps(payload, indent=2) + "\n",
encoding="utf-8",
)
return {
"action": "send",
"reason": "request changed",
"hash": current_hash,
"changed": changed,
}
def main():
if len(sys.argv) != 2:
print("usage: python3 prompt_journal.py JOURNAL.json", file=sys.stderr)
sys.exit(1)
try:
request = json.loads(sys.stdin.read())
except json.JSONDecodeError as exc:
print("invalid JSON: %s" % exc, file=sys.stderr)
sys.exit(1)
if not isinstance(request, dict):
print("request must be a JSON object", file=sys.stderr)
sys.exit(1)
result = decide(Path(sys.argv[1]), request)
print(json.dumps(result, indent=2))
if result["action"] == "block":
sys.exit(2)
if __name__ == "__main__":
main()
Save that as prompt_journal.py. The first fixture is the liar. Save it as retry.json.
{"text":"Where is the rubric?","instruction":"Label the text as question, complaint, or praise.","temperature":0}
After parsing, canonical form drops insignificant spacing and sorts keys. The intermediate string I expect is one line, and you should print it before you trust any hash.
{"instruction":"Label the text as question, complaint, or praise.","temperature":0,"text":"Where is the rubric?"}
That string, UTF-8 encoded, is what gets hashed. Not your feelings about the sentence. Not the scratch note. If canonicalize does not produce that line, stop. The rest of the lab is noise.
First contact. No journal file yet. This run is allowed to send, because there is no prior coat to compare.
rm -f journal.json
python3 prompt_journal.py journal.json < retry.json
echo "exit=$?"
I expect action to be send, reason to be no prior entry, and changed to list instruction, temperature, and text. Exit code 0. The hash field should be 64 hex characters. Copy it. Two identical payloads must print the same digest, or the receipt is not a receipt.
Second contact. Do not edit the file. This is the input that should fail the send.
python3 prompt_journal.py journal.json < retry.json
echo "exit=$?"
Expected: action is block, reason is empty diff, changed is [], exit code 2, same hash as the first run. A green exit here would be a bug. You were about to buy a photocopy and call it research.
Then one real edit, saved as retry_v2.json. I added a tie-break sentence and left the student text alone.
{"text":"Where is the rubric?","instruction":"Label the text as question, complaint, or praise. If it asks for a fact, choose question.","temperature":0}
python3 prompt_journal.py journal.json < retry_v2.json
echo "exit=$?"
Expected: send, reason request changed, and changed exactly ["instruction"]. Exit 0. A new hash. That is the first moment a model call would be a new observation rather than a nervous tic. Why would a locked request open on the fourth knock?
Now the key-order impersonator. Save this as retry_reordered.json and point it at the journal only after you have stored the first fixture again, or after you delete journal.json and store retry.json once more. Same values. Different handwriting.
{"instruction":"Label the text as question, complaint, or praise.","temperature":0,"text":"Where is the rubric?"}
rm -f journal.json
python3 prompt_journal.py journal.json < retry.json
python3 prompt_journal.py journal.json < retry_reordered.json
echo "exit=$?"
Parsed equality should block the second command. If your fork sends, you compared raw strings and let key order impersonate a hypothesis. What did you think had changed?
A non-object is the other bad input. A list is not a slightly weird dict. It is a different species, and it must not touch the journal.
echo '["not","a","request"]' | python3 prompt_journal.py journal.json
echo "exit=$?"
Expected: stderr says request must be a JSON object, exit 1. Malformed JSON, such as a lone {, should exit 1 as well, with invalid JSON: on stderr. A traceback there means you skipped the try.
Whitespace stripping felt professional and hid the bug. I normalized the instruction, the diff vanished, and the model-facing text still had the old wording. Compare the values you will actually send. Store a pretty copy if you want to read it later. Do not hash your manners.
A notes field is the same trap with a friendlier name. I almost hashed notes so I could write "fixed the tie-break" without touching instruction. That is the scratch note walking into the coat check and stealing a ticket. If you add notes, keep them out of digest. Then admit a notes-only edit must block.
Replacing journal.json by hand is how you debug a ghost. Delete it, or accept that you are no longer testing the script. And 0 versus 0.0 is a real Python inequality, so look at changed before you invent a story about the model. I am not making type trivia the lesson of the night. I am saying the keys on stdout are the evidence, not the sigh you made before hitting enter.
What should you understand when this works? A retry is not an experiment. Canonical JSON is the request with key order and insignificant spacing removed. Exit code 2 means you did not change the question. Repetition does not thaw a deterministic door.
The traced result was not a higher grade. It was a shorter night. Four "retries" collapsed into one blocked photocopy and one real instruction change. The wrong label did not become right by being asked again. Of course it didn't.
The journal never calls a model. That is intentional. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access is where I would spend the actual send, and only after action is send. The free server option is where I would rerun this same script when my laptop and the lab image disagree about Python or locale.
I have not measured quotas, model names, hardware, or how long that access lasts, so those numbers stay out of the write-up. If you remove every product mention, the receipt still works against any endpoint you already have. The guard does not know who is on the other side of the send, and it should not pretend to.
Who should skip this? You, if the study question is paraphrase. Exact equality will treat "Label" and "Classify" as different studies even when the task barely moved, and it will miss two prompts that mean the same thing. Also skip it for streaming timers, tool-call schemas, or gold-span grading. Those are other nights. This night is only the empty diff.
If a free model call and a clean machine would help you try the one send that survives the journal, use MonkeyCode's free model access for that call and the free server to rerun the receipt away from your laptop. Then tell me which fixture you predicted would block, and which one slipped through. I want the counterexample, not the applause.