# Learn Eval Ownership by Building a Tiny Lab Witness > Source: > Published: 2026-09-13 19:12:42+00:00 Last Tuesday in Halifax my laptop sounded like it wanted to leave the room. I had a 40-line name-normalization lab, a messy CSV, and a due time that was closer than my pride wanted to admit. The function was supposed to turn a display name into a join key I could match against a roster. I almost pasted the whole assignment into a remote model and called it sleep. Then a meaner question showed up. If the answer comes back from a machine I do not own, what am I turning in? A solution? Or a polite paragraph that looked like a solution under the demo string? This is a case study of one tiny project. Background, goal, implementation, results, lessons. I am not walking you through a framework. I am asking one learning question and paying for it in code: can I use a remote draft without handing the remote machine the right to grade me? I am an AI and CS student. I do not have a quiet GPU under the desk. Paid tokens are a fast way to turn a two-hour lab into a shrug, and I have bought that shrug before. The newer temptation is the opposite mood. Free remote models. A free server. Paste until the function looks finished. The public argument this week is loud. Some people say models already code better than most of us. Some people ask if the models just made us lazier. I do not have a salary survey. I have a fixture. When the model is hosted somewhere else, the fixture is the only evidence I trust. Here is the sticky-note analogy. A remote model is a photocopier in another building. You can send it a page. You should not send it the answer key and then clap when a page comes back that resembles the key. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I reached for MonkeyCode in this lab as an open-source student tool because it currently offers free model access and a free server option. I am not going to invent model names, token ceilings, hardware, or how long that free tier lasts. Those numbers go stale. A witness file on my disk does not. I wanted a local program that does four things and refuses a fifth. It keeps a hidden fixture file I wrote by hand. It sends only a constrained draft request through one transport function. It appends a witness line with hashes and pass/fail. It never prints the hidden expected output to the remote side. The fifth thing it will not do is ship roster names off the laptop. That is the honor-code line. Course data stays on disk. The remote side is allowed to see a demo string. It is not allowed to see the trap. If you want to play, guess now. Which input do you think a rushed normalizer will bless? `"Ada Lovelace"`, `"Ada Lovelace"`, or `""`? Write the guess in the margin. Then run the file. The point of the lab is not applause. The point is to be wrong in public, cheaply. I used Python 3.11, the standard library, and a terminal. No pip install. No notebook. If `python3 --version` prints 3.11 or 3.12, you can reproduce this tonight. The remote call hides behind one function on purpose. In the runnable lab below, that function is a local stand-in that mimics a rushed draft: strip the edges, lowercase, ship it. When I used a free remote server, I swapped only that function. The fixtures never left the machine. Create a folder. Put two files in it. The first file is the answer key you refuse to mail to a stranger. ``` { "cases": [ { "id": "demo", "given": "Ada Lovelace", "expected": "ada lovelace", "note": "the happy path the prompt already leaked" }, { "id": "double_space", "given": "Ada Lovelace", "expected": "ada lovelace", "note": "inner whitespace is the join-key bug" }, { "id": "empty", "given": "", "expected": "", "note": "empty should stay empty, not become None or 'none'" } ] } ``` Save that as `fixtures.json`. Now the witness. This is the whole lab. It is ugly on purpose. `exec` is a teaching knife, not a security boundary. Do not point it at the internet and walk away. ``` python # lab_witness.py from __future__ import annotations import hashlib import json from dataclasses import dataclass from pathlib import Path WITNESS_PATH = Path("witness.jsonl") FIXTURE_PATH = Path("fixtures.json") def sha256_text(text: str) -> str: return hashlib.sha256(text.encode("utf-8")).hexdigest()[:16] def load_fixtures() -> dict: return json.loads(FIXTURE_PATH.read_text(encoding="utf-8")) def remote_draft(prompt: str) -> str: """Local stand-in for a remote model. Swap this function if you have a real transport. It is intentionally weak: strip + lower, no inner-space collapse. """ if "Write a Python function named normalize_name" not in prompt: return "ERROR: unexpected prompt" return ( "def normalize_name(name: str) -> str:\n" " return name.strip().lower()\n" ) def build_prompt() -> str: # Hidden fixtures are not in this prompt. That is the whole trick. return ( "Write a Python function named normalize_name.\n" "It takes a display name and returns a join key.\n" "Demo only: normalize_name('Ada Lovelace') -> 'ada lovelace'.\n" "Do not emit tests. Do not emit extra text. Emit the function." ) def exec_function(source: str): ns: dict = {} exec(source, ns, ns) # lab-sized, not a sandbox fn = ns.get("normalize_name") if not callable(fn): raise ValueError("no normalize_name in draft") return fn @dataclass class Row: case_id: str given: str expected: str actual: str | None ok: bool note: str def run_cases(fn, fixtures: dict) -> list[Row]: rows = [] for case in fixtures["cases"]: given = case["given"] expected = case["expected"] try: actual = fn(given) ok = actual == expected except Exception as exc: actual = f"EXC:{type(exc).__name__}" ok = False rows.append(Row(case["id"], given, expected, actual, ok, case["note"])) return rows def append_witness(prompt: str, source: str, rows: list[Row]) -> None: record = { "prompt_sha": sha256_text(prompt), "draft_sha": sha256_text(source), "passed": [r.case_id for r in rows if r.ok], "failed": [r.case_id for r in rows if not r.ok], } with WITNESS_PATH.open("a", encoding="utf-8") as fh: fh.write(json.dumps(record) + "\n") def main() -> None: fixtures = load_fixtures() prompt = build_prompt() source = remote_draft(prompt) print("=== draft from remote_draft() ===") print(source) fn = exec_function(source) rows = run_cases(fn, fixtures) print("=== local fixture results ===") for r in rows: flag = "PASS" if r.ok else "FAIL" print( f"{flag} {r.case_id}: given={r.given!r} " f"expected={r.expected!r} actual={r.actual!r}" ) if not r.ok: print(f" why this exists: {r.note}") append_witness(prompt, source, rows) failed = [r for r in rows if not r.ok] if failed: print(f"=== lab incomplete: {len(failed)} fixture(s) still own you ===") else: print("=== all local fixtures passed ===") if __name__ == "__main__": main() ``` Run it like a boring lab, not like a demo day. ``` python3 --version python3 lab_witness.py cat witness.jsonl ``` On my machine the stand-in draft looks finished. That is the trap. The demo string is already in the prompt, so of course the photocopier can copy it. The empty string also survives, because `.strip()` on `""` is still `""`. The double space is the one that keeps the lab honest. ``` php === draft from remote_draft() === def normalize_name(name: str) -> str: return name.strip().lower() === local fixture results === PASS demo: given='Ada Lovelace' expected='ada lovelace' actual='ada lovelace' FAIL double_space: given='Ada Lovelace' expected='ada lovelace' actual='ada Lovelace' why this exists: inner whitespace is the join-key bug PASS empty: given='' expected='' actual='' === lab incomplete: 1 fixture(s) still own you === ``` Did you guess `empty`? Plenty of people do. Empty feels cursed. Empty is also the case a `.strip()` implementation accidentally gets right. The join-key bug is the inner gap. Two spaces in a CSV name is not an edge case in a student roster. It is Tuesday. The witness line is the part I actually kept. It does not store the roster. It stores hashes and the list of case ids that still own me. If I change the prompt later, the prompt hash moves. If a remote draft suddenly “passes,” I can ask whether I leaked a fixture into the prompt. That question has saved me from turning in a function I could not explain in office hours. A leaked prompt looks like this, and I have written this prompt when I was tired. Do not run this version if you still want the lesson. ``` php def build_prompt_leaked(fixtures: dict) -> str: # This is the photocopier-with-the-answer-key version. return "Hidden cases: " + json.dumps(fixtures) ``` Once the remote side sees `double_space`, a pass stops meaning “I understand join keys.” It starts meaning “the machine was shown the exam.” That is not eval. That is a spoiler. The draft was not evil. It was local-minimum helpful. `.strip().lower()` is what I would type in the first ninety seconds too. The failure is not that a model is bad at strings. The failure is that a demo string is a terrible grader. If I had stopped at the first green line, I would have submitted a join key that duplicates people. Think of the witness as a hallway monitor. The remote function can still be useful. It can still draft. It just does not get to stamp the hallway pass. I own the ugly input. I own the log. I own the sentence I would say if a TA asked why two spaces matter. Common mistakes showed up as soon as I tried to “improve” the lab. People paste the fixture file into the prompt because they want a green run more than they want a true run. People treat a vendor log as their notebook, then lose the thread when the chat disappears. People call `exec` a sandbox. It is not. This file is a desk experiment. If the draft ever came from a truly untrusted channel, I would not exec it. I would read it. Another mistake is subtler. You add more demo strings until the model memorizes your style, and you call that robustness. That is still a spoiler, just a longer one. A fixture you refuse to transmit is a different kind of test. It is closer to how a marker will actually grade you. After this lab I want a reader to be able to say three sentences without reaching for a slogan. A remote draft is a photocopier. A local fixture is a grader. If those two live in the same prompt, you are grading the photocopier on handwriting it already saw. I would not use this approach on a take-home exam if the syllabus forbids outside models. I would not send real student names, emails, or health flags to any remote box, free or not. I would not pretend a free server is a production SLA. Free access is a door, not a contract. I also would not use `exec` outside a folder I am willing to delete. If you already keep a local test suite and you never paste the tests into the model, you do not need my theater. You already own eval. This article is for the night you almost didn’t. The extension is one extra case, and I mean one. Add a non-breaking space, `"Ada\u00a0Lovelace"`, and decide what your join key should be. Predict the stand-in result before you run it. If your draft still passes the demo and fails the new case, the witness did its job. If you “fix” it by leaking the new case into the prompt, the witness will still record a pass, and you will have taught yourself the wrong lesson. I am a student. I like tools that let me iterate without lighting money on fire. If you want to try the same split I used in this case study — local witness, remote draft, fixtures that never leave disk — MonkeyCode’s free model access and free server option are the remote side I plugged into that one transport function. Keep the ugly inputs on your laptop. That is the whole assignment.