{"slug": "if-the-remainder-doesn-t-shrink-it-s-a-zero-a-bootcamp-lab-on-agent-loop", "title": "If the Remainder Doesn't Shrink, It's a Zero: A Bootcamp Lab on Agent Loop Progress", "summary": "A developer has published a bootcamp lab that requires coding agents to log a typed, comparable \"remainder\" of unfinished work each iteration and fail unless that remainder strictly shrinks. The lab defines allowed remainder kinds such as unresolved_files, unresolved_assertions, open_todos, and schema_violations, and enforces them through a frozen test oracle and an NDJSON progress contract. The developer argues that token budgets, halt contracts, and traces do not prove an agent loop actually made progress.", "body_md": "Your agent can thrash for forty turns and still be stuck. Busy is not progress. In this lab you fail unless every iteration names a remainder and that remainder strictly shrinks.\n\nI do not grade the model's vibe. I grade a number that goes down.\n\nWhy this lab, and why now? Coding agents look productive because they emit files, tool calls, and confident summaries. That is theater. If the work left to do does not get smaller, you built a spinner with extra logging.\n\nA student ships an agent that \"refactors the repo.\" It lists files. It rereads the same module. It rewrites a comment. Then it lists files again. The transcript is long. The remainder is identical.\n\nSound familiar? That loop is not searching. It is pacing.\n\nHalt contracts tell you when to stop. Token budgets tell you what the run cost. Traces tell you what happened. None of those prove the loop moved. This lab is the missing piece: a progress contract.\n\nA remainder is a typed, comparable quantity of unfinished work. Not a paragraph. Not \"almost done.\" A number plus a unit plus a proof rule.\n\nPick one remainder kind for the assignment. Stick with it for the whole run.\n\n`unresolved_files`: count of files still failing the assignment tests` unresolved_assertions`: count of failing asserts in a frozen test file I provide` open_todos`: count of `TODO(lab)` markers the agent is allowed to create, then must delete`schema_violations`: count of JSON records that fail a schema you freeze in `schema.json`\nIf you cannot compare remainder `n` with remainder `n-1`, you do not have a remainder. You have a diary.\n\nYou need Node 20+, git, and a throwaway directory. Do not run this against a real repo. The agent will write files. That is the point of a sandbox, not a personality trait.\n\n```\nmkdir agent-remainder-lab && cd agent-remainder-lab\ngit init\nnpm init -y\nmkdir src tests fixtures\n```\n\nFreeze the human oracle first. The agent does not get to rewrite the tests that define \"done.\" If the model authors the exit condition, the remainder can shrink by cheating. We already buried that lesson. Not this week.\n\n``` python\ncat > tests/oracle.test.mjs <<'EOF'\nimport test from 'node:test';\nimport assert from 'node:assert/strict';\nimport { readFileSync, existsSync } from 'node:fs';\n\ntest('inventory.json exists and parses', () => {\n  assert.equal(existsSync('fixtures/inventory.json'), true);\n  JSON.parse(readFileSync('fixtures/inventory.json', 'utf8'));\n});\nEOF\n```\n\nSeed a broken fixture so the first remainder is honest.\n\n```\nprintf 'not-json\\n' > fixtures/inventory.json\nnode --test tests/oracle.test.mjs; echo \"oracle exit: $?\"\n```\n\nThat failing oracle is your starting remainder. Write it down before the agent starts. If you skip the baseline, later \"progress\" is fan fiction.\n\nEvery agent iteration must append one JSON line to `remainder.ndjson`. No pretty print. No extra keys the harness does not know. If the file is missing, the run is a zero.\n\n``` js\n// remainder-contract.mjs\nexport const ALLOWED_KINDS = new Set([\n  'unresolved_files',\n  'unresolved_assertions',\n  'open_todos',\n  'schema_violations',\n]);\n\nexport function parseLine(line, lineno) {\n  let row;\n  try {\n    row = JSON.parse(line);\n  } catch {\n    throw new Error(`line ${lineno}: not JSON`);\n  }\n  for (const key of ['iteration', 'kind', 'value', 'unit', 'note']) {\n    if (!(key in row)) throw new Error(`line ${lineno}: missing ${key}`);\n  }\n  if (!Number.isInteger(row.iteration) || row.iteration < 1) {\n    throw new Error(`line ${lineno}: iteration must be an integer >= 1`);\n  }\n  if (!ALLOWED_KINDS.has(row.kind)) {\n    throw new Error(`line ${lineno}: unknown kind ${row.kind}`);\n  }\n  if (!Number.isInteger(row.value) || row.value < 0) {\n    throw new Error(`line ${lineno}: value must be an integer >= 0`);\n  }\n  if (typeof row.unit !== 'string' || row.unit.length === 0) {\n    throw new Error(`line ${lineno}: unit must be a non-empty string`);\n  }\n  return row;\n}\n\nexport function assertProgress(rows) {\n  if (rows.length === 0) throw new Error('no remainder rows');\n  const kind = rows[0].kind;\n  const unit = rows[0].unit;\n  let prev = Number.POSITIVE_INFINITY;\n  let stalled = 0;\n  for (const row of rows) {\n    if (row.kind !== kind) throw new Error('kind changed mid-run');\n    if (row.unit !== unit) throw new Error('unit changed mid-run');\n    if (row.value < prev) {\n      stalled = 0;\n      prev = row.value;\n      continue;\n    }\n    stalled += 1;\n    if (stalled > 1) {\n      throw new Error(`no shrink for ${stalled} consecutive iterations at ${row.value} ${unit}`);\n    }\n  }\n  return { kind, unit, start: rows[0].value, end: rows[rows.length - 1].value, iterations: rows.length };\n}\n```\n\nOne stall is allowed. Two consecutive non-shrinks are a fail. Why the slack? Tools flake. Networks hiccup. I am not grading luck. I am grading whether the loop can recover and still reduce the remainder.\n\n``` js\n// remainder-harness.mjs\nimport { readFileSync } from 'node:fs';\nimport { parseLine, assertProgress } from './remainder-contract.mjs';\n\nconst lines = readFileSync('remainder.ndjson', 'utf8')\n  .split('\\n')\n  .map((l) => l.trim())\n  .filter(Boolean);\n\nconst rows = lines.map(parseLine);\nconst summary = assertProgress(rows);\nif (summary.end !== 0) {\n  throw new Error(`remainder ended at ${summary.end} ${summary.unit}, expected 0`);\n}\nconsole.log(JSON.stringify(summary, null, 2));\n```\n\nRun it like a grader, not like a demo.\n\n```\nprintf '%s\\n' \\\n  '{\"iteration\":1,\"kind\":\"unresolved_assertions\",\"value\":3,\"unit\":\"asserts\",\"note\":\"oracle red\"}' \\\n  '{\"iteration\":2,\"kind\":\"unresolved_assertions\",\"value\":3,\"unit\":\"asserts\",\"note\":\"reread same file\"}' \\\n  '{\"iteration\":3,\"kind\":\"unresolved_assertions\",\"value\":1,\"unit\":\"asserts\",\"note\":\"fixed parse\"}' \\\n  '{\"iteration\":4,\"kind\":\"unresolved_assertions\",\"value\":0,\"unit\":\"asserts\",\"note\":\"oracle green\"}' \\\n  > remainder.ndjson\n\nnode remainder-harness.mjs\n```\n\nSwap the last non-zero pair so values go `3, 3, 3` and watch it throw. That is the whole lab. A spinner should hurt.\n\nDo these in order. Skipping ahead is how people invent a remainder after the fact.\n\n`kind` and `unit` in `LAB.md`. Changing them later is a rewrite of the assignment.` blocked` stop with a reason, or the harness fails you.`value: 0` is accepted only if `node --test tests/oracle.test.mjs` exits 0 on a clean checkout of your submission.\nCheckpoint 3 is where most people cheat. They let the model narrate progress, then they type a shrinking sequence. I rerun the oracle. Your NDJSON is not evidence unless it matches the test runner.\n\nHere is a skeleton. It is a lab wrapper. It is not a product.\n\n``` js\n// loop.mjs — proposed student wrapper, not a claimed production agent\nimport { appendFileSync } from 'node:fs';\nimport { spawnSync } from 'node:child_process';\n\nfunction oracleFailures() {\n  const r = spawnSync('node', ['--test', 'tests/oracle.test.mjs'], { encoding: 'utf8' });\n  const fails = (r.stderr + r.stdout).match(/not ok /g);\n  return fails ? fails.length : (r.status === 0 ? 0 : 1);\n}\n\nfunction record(iteration, value, note) {\n  const row = {\n    iteration,\n    kind: 'unresolved_assertions',\n    value,\n    unit: 'asserts',\n    note,\n  };\n  appendFileSync('remainder.ndjson', JSON.stringify(row) + '\\n');\n}\n\n// You plug your tool-calling agent into step().\n// step() may edit files. It may not edit tests/oracle.test.mjs.\nasync function run({ maxIters = 8, step }) {\n  let prev = oracleFailures();\n  record(1, prev, 'baseline');\n  let stalled = 0;\n  for (let i = 2; i <= maxIters; i++) {\n    await step(i, prev);\n    const now = oracleFailures();\n    record(i, now, `after step ${i}`);\n    if (now === 0) return { ok: true, iteration: i };\n    if (now < prev) stalled = 0;\n    else stalled += 1;\n    if (stalled > 1) {\n      throw new Error('remainder did not shrink; stop instead of spinning');\n    }\n    prev = now;\n  }\n  throw new Error('hit maxIters with remainder > 0');\n}\n\nexport { run, oracleFailures };\n```\n\nNotice what is missing. No retry storm. No \"one more pass.\" The loop is allowed to work only while the remainder falls. That is harsh. It is also how you keep a weekend lab from becoming an unbounded cloud bill.\n\nYou can implement `step()` with any model that can edit files. The grade is the harness, not the vendor.\n\nDisclosure: This article was prepared as part of MonkeyCode's product outreach.\n\nI mention [MonkeyCode](https://monkeycode.ai) here for one lab reason: students who do not want to paste personal API keys into a bootcamp wrapper can use its free model access and the free server option to run `step()` off their laptop. That is convenience. It is not extra credit. If the remainder file is fake, the free server will not save you.\n\nDo not treat the remote box as a second brain. Treat it as a machine that still has to leave `remainder.ndjson` and a green oracle in the repo I clone.\n\nI grade artifacts I can rerun. I do not grade screenshots of a chat.\n\n| Score | What I clone | Remainder rule | Oracle | \n|---|---|---|---|\n| 0 | Missing `remainder.ndjson` , or tests edited | n/a | n/a | \n| 1 | NDJSON parses, kind locked | Values wander or stall twice | Still red | \n| 2 | NDJSON monotonic with one stall max | Ends above 0 | Red or flaky | \n| 3 | Harness exits 0 | Ends at 0 | Green on my machine | \n| 4 | Score 3 plus a `blocked` path | Declares block instead of spinning | Green or honest block | \n\nStretch does not replace a broken remainder. A beautiful agent that cannot shrink a number is still a 1.\n\nOnly after a 3.\n\n`node_modules`, recopy fixtures, rerun `loop.mjs`. Same remainder sequence shape, same final zero. Hidden local caches are a different lab. Do not smuggle them back in.`step()` a tool result that claims the oracle is green when it is not. The wrapper must remeasure with `oracleFailures()`, not trust the tool text.\nThat third stretch is the one I actually care about. Models will believe their own tools. Your wrapper should not.\n\nThis remainder model is crude. Some real work is non-monotone: you add a failing test on purpose, remainder jumps, then it falls. I ban that jump in the core lab because I cannot fairly grade \"strategic getting worse\" in a two-hour window. If your research loop needs exploratory regressions, this rubric will punish you. Use a different contract.\n\nSkip this lab if you are driving production incident response. A shrinking unit-test count is not a blast radius. Skip it if your agent only answers questions and never touches files. There is no remainder to shrink. Skip it if you cannot freeze an oracle. Then you are grading prose again.\n\nAlso skip any remainder you cannot recompute from the repo. \"Confidence: 0.81\" is not a remainder. I cannot rerun your model's self-esteem.\n\nKeep it short. I read the NDJSON first.\n\nIf your writeup is longer than the remainder file, you are hiding. Cut it.\n\nAn agent loop without a shrinking remainder is a `while (true)` with better copy. Name the leftover work. Measure it after every step. Stop when it hits zero, or when it refuses to fall.\n\nThat is the whole grade. Everything else is narration.", "url": "https://wpnews.pro/news/if-the-remainder-doesn-t-shrink-it-s-a-zero-a-bootcamp-lab-on-agent-loop", "canonical_source": "https://dev.to/hackjs_7468/if-the-remainder-doesnt-shrink-its-a-zero-a-bootcamp-lab-on-agent-loop-progress-3p4e", "published_at": "2026-09-12 18:17:00+00:00", "updated_at": "2026-09-12 18:49:35.060343+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "mlops"], "entities": ["Node.js", "Git", "npm"], "alternates": {"html": "https://wpnews.pro/news/if-the-remainder-doesn-t-shrink-it-s-a-zero-a-bootcamp-lab-on-agent-loop", "markdown": "https://wpnews.pro/news/if-the-remainder-doesn-t-shrink-it-s-a-zero-a-bootcamp-lab-on-agent-loop.md", "text": "https://wpnews.pro/news/if-the-remainder-doesn-t-shrink-it-s-a-zero-a-bootcamp-lab-on-agent-loop.txt", "jsonld": "https://wpnews.pro/news/if-the-remainder-doesn-t-shrink-it-s-a-zero-a-bootcamp-lab-on-agent-loop.jsonld"}}