Why osFoundry makes Claude Code name the commit when it asks to rerun a GitNest check

OsFoundry's GitNest MCP endpoint requires Claude Code to supply six fields—including an expected change-set version and composition OID—when proposing a check rerun, so approval targets the exact commit the reviewer saw rather than a recomposed change set. A simulation script, checks_sim.py, shows that common model shortcuts for judging check status are wrong in 5 of 7 cases for a no-failure heuristic and 1 of 7 for a passed-equals-total ratio, while a port of GitNest's summarise function handles empty, queued, timed-out and unknown statuses correctly.

This is the tool Claude Code calls when it wants a failed GitNest check run again, as osFoundry https://osfoundry.io 's live MCP endpoint lists it in tools/list no token needed for the catalogue : { "name": "propose gitnest check rerun", "description": "Ask the repository owner to retry one finished check on a specific composed change-set version. Approval rechecks repository access, version, commit and check state.", "inputSchema": { "type": "object", "properties": { "repository id": {"type": "string"}, "change set id": {"type": "string"}, "check name": {"type": "string", "maxLength": 200}, "expected change set version": {"type": "integer", "minimum": 1}, "expected composition oid": {"type": "string"}, "idempotency key": {"type": "string", "minLength": 8, "maxLength": 128} }, "required": "repository id", "change set id", "check name", "expected change set version", "expected composition oid", "idempotency key" , "additionalProperties": false }, "annotations": {"readOnlyHint": false, "openWorldHint": false, "destructiveHint": true}, "securitySchemes": {"type": "oauth2", "scopes": "osfoundry:write" } } Its read-side partner takes two arguments: list gitnest checks {repository id, change set id} , "bounded check status for the current composed commit of a change set". Two more reads feed the decision: list gitnest saved results limit 1–50 and read gitnest work kind is tasks , changes or runs . All three are osfoundry:read with readOnlyHint: true . Six required fields to rerun one check is a lot. A CI button needs a job ID. The extra two, a version and a commit, are what this post is about. No claude binary is installed on this Mac, so checks sim.py plays the client. The code and its full output are in code/ . Before an agent proposes anything, it has to read the check list correctly. osFoundry's GitNest UI summarises a commit's check runs with one small pure function summarise in check-runs.ts . Its order of tests is the rule: none , never passed . failure , timed out or cancelled is failed . success keeps it running , including a status the client doesn't recognise. The script ports that rule and compares it with two shortcuts a model writes on its own: == verdict == case summarise no-failure ratio no checks none green green all passed passed 2 green green one queued running 1/2 green red one failed failed: e2e red red one timed out failed: e2e green red one cancelled failed: e2e green red unknown status running 1/2 green red shortcut wrong: no-failure 5/7, ratio 1/7 "Nothing failed, so it's green" is wrong in 5 of 7 cases. It calls an unexamined commit green, a queued check green and a timeout green. The ratio check passed == total is closer, but on zero checks 0 == 0 and it says green too. That's the case the GitNest code comments call the most misleading one, since a commit with no checks hasn't been verified at all. The UI renders it as "No checks have run against this commit", with no tick. If you prompt Claude Code to answer "is it green?", give it these four rules in CLAUDE.md . The tool returns statuses, not a verdict. Because the person approving it is looking at a specific commit, and the change set can move before they click. A change set in GitNest is composed from saved results into one commit. Recompose it, after a new saved result or a moved base, and it gets a new version and a new composed commit, and its checks start over. A rerun proposal waits for a human. If it only said "rerun e2e on cs 4e81a0", approval would rerun e2e on whatever the change set is by then, which may be a commit the owner never saw. checks sim.py models that wait. Each trial is one proposal. The agent reads version 3, commit A, with e2e failed. The owner's review time is log-normal with a median of 8 minutes. That's an assumption, not a measurement. During the wait, the change set is recomposed as a Poisson process, on average every 45 minutes, and after a recompose the checks take 4 minutes to run. In 10% of trials a second Claude Code session read the same failure and filed its own proposal with its own key, and the owner approves both. Three approval-time guards, 10,000 trials each: == approval-time guards n=10000, review median 8 min, recompose mean every 45 min, 10% second session == guard ran on reviewed commit ran on unreviewed commit duplicate run refused stale refused not finished none 7789 1411 1764 0 0 version+oid 7789 0 740 2435 0 full 7789 0 0 2435 740 With no guard, 1,411 approvals reran a check on a commit the owner hadn't reviewed, and 1,764 queued a check that was already queued or running. Comparing version and commit at approval drops the unreviewed runs to 0 and refuses 2,435 stale cards instead. The 740 duplicates left are the second sessions: same version, same commit, but the first approval already requeued the check. The third guard, "is this check still finished?", catches those. That's the full set the tool description lists: "Approval rechecks repository access, version, commit and check state." The recompose rate is the input I'm least sure of, so here's the no-guard row at three rates: == sensitivity: unreviewed-commit runs with guard 'none' == recompose every 15 min unreviewed 2449 duplicates 3128 recompose every 45 min unreviewed 1411 duplicates 1764 recompose every 180 min unreviewed 452 duplicates 1208 Even a change set that moves every three hours gives 452 wrong-commit runs per 10,000. The guard costs nothing on the happy path: the reviewed-commit count is 7,789 in all three rows. In this model they always move together, so either alone would score the same. They answer different questions, though. The version is the change set's own counter, and it moves when its contents change. The commit is what the checks actually ran on, and it's what the card prints as "Reviewed commit: …". Asking the client to send both means the server can refuse on either, and the owner reads a commit hash on the card, not an internal counter. The schema enforces the version as an integer ≥ 1. The commit is a free string, so its check happens on the server. The script validates arguments against the live inputSchema with jsonschema : == propose gitnest check rerun arguments == ok PASS no oid REJECT 'expected composition oid' is a required property version 0 REJECT 0 is less than the minimum of 1 version as string REJECT '3' is not of type 'integer' two checks REJECT 'e2e', 'lint' is not of type 'string' check 201 chars REJECT 'ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc key 7 chars REJECT 'abcdefg' is too short extra 'force' REJECT Additional properties are not allowed 'force' was unexpected required: repository id, change set id, check name, expected change set version, expected composition oid, idempotency key Two rejections are design, not hygiene. check name is a string, not a list, so "rerun everything" is one card per check and the owner decides each one. And additionalProperties: false means a model that invents "force": true gets an error instead of a silently ignored flag. The idempotency key should come from what the proposal is about, not from the attempt. The script uses "rerun-" + sha256 change set id, version, oid, check name :32 : key retry same: True key after recompose same: False len 38 A retry after a lost response sends the same key and doesn't make a second card. A recompose changes the key, so a proposal for the new commit is a new card, which is right. == tools/list == 59 tools, 48140 B via json.dumps, 12722 tok cl100k list gitnest checks 612 B 158 tok readOnly=True destructive=False scope=osfoundry:read list gitnest saved results 707 B 182 tok readOnly=True destructive=False scope=osfoundry:read read gitnest work 685 B 189 tok readOnly=True destructive=False scope=osfoundry:read propose gitnest check rerun 1011 B 268 tok readOnly=False destructive=True scope=osfoundry:write all 13 gitnest tools 11227 B 3009 tok The 13 GitNest tools are about a quarter of the catalogue's tokens counted with tiktoken cl100k base , so treat them as approximate . Ten unauthenticated calls each from this Mac in Japan to the live endpoint: bash $ python3 probe.py initialize HTTP 200 308 B median 566 ms min 326 max 807 capabilities: {"tools": {"listChanged": false}} tools/list HTTP 200 45368 B median 722 ms min 420 max 1406 tools/call list gitnest checks HTTP 401 337 B median 552 ms min 409 max 903 The 401 carries error description="Connect osFoundry to continue" and a resource metadata URL, which is what sends Claude Code into its OAuth flow. initialize advertises no push, so nothing tells the client the retry was approved or that the check finished. Waiting for a check is a runner's work on your machine plus a human's click, minutes rather than milliseconds. The cheap pattern is to end the turn after proposing and call list gitnest checks once when the user says they've approved. Not osFoundry. GitNest's runner panel says checks "run on your machine, not ours". A registered runner polls with its own token OSF RUNNER TOKEN=… osfoundry-runner --api https://api.osfoundry.io , and without one checks "stay queued". Approving a retry only requeues a check. Claude Code never holds the runner token, and the read tools leave out local bindings and credentials. The agent can see that e2e failed. It can't make anything execute without you and your runner. The change-set review page with the checks panel is on the GitNest page https://osfoundry.io/platform/gitnest . For the protocol itself, see the MCP glossary entry https://osfoundry.io/glossary/mcp .