{"slug": "bounded-checking-and-repair-for-ai-or-other-proposers", "title": "Bounded checking and repair for AI(or other) proposers", "summary": "A developer has published BRACE, an open-source harness that pairs small, domain-specific verifiers with AI or program-generated proposals so that each computation is either checked against explicit acceptance rules or explicitly refused. In the PAT example, a mislabeled lesson proposing addition where the instruction calls for multiplication is recorded as `different-operation` and the answer is withheld, even though a check on the final number alone would have passed. The first five demo commands run offline on Python 3.12 or later using only the standard library, with no model download or account required, and the connector example suite produces 14 checked mappings, two verified no-matches and two refusals while applying no changes.", "body_md": "BRACE provides small, domain-specific verifiers for developers who need checked computations and repair previews. Each domain declares its supported inputs, acceptance rules and work limits. It returns a checked result or an explicit refusal. Direct code handles work the contract already determines; the integer harness also accepts proposals from an AI or another program.\n\nAt an input of 2, `2 + 2` and `2 × 2` both give **4**. In the PAT example, a mislabeled lesson proposes addition when the instruction calls for multiplication. A check on the final number alone would pass. BRACE records `different-operation` and withholds the answer. [Run the PAT example](#try-pat-as-a-proposer)\n\n| Capability | Try it | Boundary | \n|---|---|---|\n| Checked integer steps and bounded retries | `python -B demo.py --json` | Supported add-then-multiply instructions; host-owned release. | \n| Connector selection and repair preview | `python -I -S -B domains/connector/demo.py` | Four supplied mappings, explicit selection rule and a fixed MockServer-to-WireMock subset; read-only. | \n| Bounded composition and accepted lessons | `python -I -S -B domains/composition/demo.py` | Four-example integer plans; explicit host acceptance, checked reuse, reopening and rollback. | \n| Read-only receipt inspection | `python -I -S -B examples/receipt_inspector/demo.py` | Exact lookups in bounded supplied captures; returns original evidence without rechecking or authorizing it. | \n| Geometry reconstruction | `python -I -S -B preview/verify.py` | The supplied bracket measurements and listed alternatives. | \n| Learned local rules and informative observations | [Research replay commands](https://github.com/Onescale414/BRACE-Harness/blob/main/research/domain-induction/README.md#reproduce) | Finite hypothesis families; reports include errors when the true rule is missing. | \n\nThe first five commands run offline with Python 3.12 or later and the standard library. No model download or account is required. Research replays also document their own assumptions and budgets; they are not a general reasoning guarantee.\n\nThe new connector command independently checks **which mapping was selected**,\nthen binds that choice to the exact source and target before building and auditing\nthe complete patch. A correct conversion alone cannot establish that the right\nmapping was chosen. The example suite produces 14 checked mappings, two verified\nno-matches and two refusals. It applies no changes.\n\n[Connector contract and runnable CLI](https://github.com/Onescale414/BRACE-Harness/blob/main/domains/connector/README.md) ·\n[How bounded domains fit together](https://github.com/Onescale414/BRACE-Harness/blob/main/docs/DOMAINS.md)\n\nThe [composition example](https://github.com/Onescale414/BRACE-Harness/blob/main/domains/composition/README.md) supplies a wrong identity\nplan for four squaring examples. A full check rejects it; bounded deterministic\nrepair finds `square` on the second check. A separate host action accepts that\nchecked draft as `Fold`. After accepting `Nudge` as `add1`, the host can request\n`do Nudge then Fold`, check the ordered plan and explicitly save it as `Lift`.\n\nReopening independently rechecks all saved lessons. Reusing `Lift` on fresh\nexamples requires another check; reversing the named order is withheld without\nchanging the lessons. The one-command demo also rehearses rollback and a bounded\nclarification. It uses disposable synthetic storage and no model calls.\n\nA checked lesson means the plan matches its supplied observations. Reuse requires new expected outputs; it does not establish an unrestricted learned rule or certify unknown predictions. The host owns acceptance, observations and storage. The CLI accepts structured events, not unrestricted chat.\n\nTo inspect captured outcomes, the optional [receipt inspector](https://github.com/Onescale414/BRACE-Harness/blob/main/examples/receipt_inspector/README.md)\nfinds the latest two records matching an exact action, status and decision. It\nreturns original request/receipt text with source identity and event positions.\nScan is the default; cache and index options are explicit. Unsupported captures\nreceive a structured refusal. This is a read-only evidence viewer, not another\nchecker or a lesson-acceptance path.\n\nA caller says, “The counter reads 13. Bump it up by 4, then triple the new reading.” Reverse those last two operations and the answer changes from **51** to **43**. Qwen 2.5 0.5B submitted the same calculation both times. BRACE verified the submitted arithmetic both times. What evidence would justify accepting the model’s interpretation?\n\nIn an initial trial, Qwen alone answered **1 of 8** clear requests correctly. Having Qwen translate the request and handing off the arithmetic raised that to **4 of 8**. The backend answered **8 of 8** when given the intended inputs. On a fresh set, a stricter reader prevented all 11 wrong or unwarranted releases, but it also withheld all eight clear requests. You can reproduce both results and try confirming or correcting a handoff yourself. [Run the handoff experiment](https://github.com/Onescale414/BRACE-Harness/blob/main/research/arithmetic-handoff/README.md)\n\nAn exploratory replay on those same 16 conversations suggested interrupting only on an explicit conflict. On a subsequent, preregistered test of **32 new conversations**, the verifier could not resolve any of them. It sent no conflict notices. The reminder alone and the conflict-only policy both got **8 of 16** clear requests right and released **20** wrong or unwarranted numbers overall. [See the prospective test and replay](https://github.com/Onescale414/BRACE-Harness/blob/main/research/arithmetic-handoff/README.md#did-conflict-only-feedback-help-on-new-conversations)\n\nA follow-up rule asked which operation came first in the original bonus example, then missed all eight fresh cases with the same ambiguity. A separate probe showed why the handoff matters. The reader dropped one of two requested additions, and BRACE correctly checked the shortened calculation. [See the missing-order trial and counterexample](https://github.com/Onescale414/BRACE-Harness/blob/main/research/arithmetic-handoff/README.md#can-the-reader-ask-which-operation-goes-first)\n\n- **Require a check before using an answer.** Route the task through`run_bound` and let the host application release the value only when`outcome.status == \"checked\"` . The application must own that release path. The proposer adapter runs as trusted Python code.\n- **Offer a check when an AI asks for one.** Expose the same call as a tool. The caller can inspect the receipts, revise a rejected step and submit it again under the host's retry policy. Skipping the call produces no BRACE verdict.\n\nThe integer API supports adding an integer and then multiplying by an integer. It can do the arithmetic directly. Plugging in your own proposer lets you measure which steps it gets wrong and whether a retry helps. BRACE checks the operation and five witness values, and the second step must cite the receipt for the accepted first step. The connector profile uses a direct selector because it outperformed the model in the bounded comparison. These are separate domain APIs; a new task type needs its own contract and checker.\n\nThe connector experiment asks whether the caller requested the action that passed the technical check. Its offline replay includes a valid patch preview for a request the caller explicitly excluded. You can reproduce the failure and inspect the evidence. [Try the connector replay](https://github.com/Onescale414/BRACE-Harness/blob/main/research/connector-clarification/README.md)\n\nFrom this directory, use Python 3.12 or later. The built-in demo and tests run offline using the standard library.\n\n```\npython -B demo.py --json\npython -B -m unittest -q test_harness\n```\n\nThe demo runs `Start with 3. Add 1, then double the result.` It tries a wrong operation first at each step. Four checks yield the final value **8**. Changing `Start` to `Begin` returns `unsupported-input`. The suite covers 18 cases. [Demo](https://github.com/Onescale414/BRACE-Harness/blob/main/demo.py) · [Tests](https://github.com/Onescale414/BRACE-Harness/blob/main/test_harness.py)\n\nBoth operands may be negative. For `Start with 6. Add -11, then multiply by -3.`, the checked values are `-5` after addition and `15` after multiplication.\n\nExtract the release and put your script beside `harness.py`. Import `Candidate`, `Reply`, `encoded`, and `run_bound` from `harness`. Call `run_bound(instruction.encode(\"utf-8\"), adapter)`. The host calls your trusted synchronous Python function `adapter(ticket, view)` for each step.\n\nReturn `Reply(ticket.ticket_id, (Candidate(encoded(payload)),))` to submit one candidate. Build `payload` as a Python dictionary with exactly these nine keys.\n\n| Keys | Values | \n|---|---|\n| `id` ,`run` ,`snapshot` | Choose a candidate ID unique within the run, such as `f\"op-{view.step}\"` for one candidate per step. Copy`ticket.run` and`ticket.snapshot` into the other two fields. | \n| `step` ,`parent` | Copy `view.step` and`view.parent` . At step 0 the parent is`None` . At step 1 the host supplies the first accepted receipt's ID. | \n| `a` ,`b` | Each operation maps `x` to`a*x+b` . At step 0 use`(1, view.snapshot.add)` . At step 1 use`(view.snapshot.multiply, 0)` . The host has already parsed both operands, including a factor of 2 for “double the result.” | \n| `witness` ,`result` | Send a list of the five outputs of `a*x+b` for inputs`[-2, -1, 0, 1, 2]` , in that order. Send the output for`x = view.before` as`result` . | \n\nCandidate IDs accept 1–64 ASCII letters, digits, underscores, periods, or hyphens. Coefficients, witness outputs, and results must be integers with absolute value at most `10**15`. The returned outcome exposes `outcome.status`, `outcome.value`, and `outcome.receipts`. Each receipt has a `step`, `verdict`, and `value`. A successful two-step run has status `checked` and a checked receipt for each step.\n\nDownload and unzip [BRACE-Harness](https://github.com/Onescale414/BRACE-Harness/archive/refs/heads/main.zip) and [PAT](https://github.com/Onescale414/PAT/archive/refs/heads/main.zip) into the same parent directory. From inside `BRACE-Harness-main`, run these commands with Python 3.12 or later. If your PAT directory has another name, pass its path to `--pat`.\n\n```\npython -m pip install \"numpy>=1.24\"\npython -B examples/pat/demo.py --pat ../PAT-main\n```\n\nEach case starts with its own in-memory PAT lessons. In the first case PAT recalls “multiplying” as multiplication and BRACE releases **4**. In the second case that word was deliberately labeled addition. At an input of 2, both adding 2 and multiplying by 2 give 4. BRACE returns `different-operation` and withholds the answer. On one retry, the adapter excludes addition only at the rejected step. PAT recalls “doubling,” BRACE rechecks both steps and releases **4**. The original lesson is still available afterward.\n\nIn the third case PAT recalls multiplication correctly, and the demo deliberately changes the reported result from **4** to **5**. BRACE returns `wrong-result` and withholds the answer. On one retry, the adapter takes the rejected draft and recalculates `2 × 2` from its original coefficients and the checked input. BRACE releases **4**. The demo compares both submitted drafts and verifies that the lesson, operation, operand and five witness values match. The new run has its own run ID and first-step receipt.\n\nThe fourth case gives the whole request one retry. PAT recalls “adding” as multiplication, so step one proposes `1 × 1 = 1`. BRACE returns `different-operation` and stops before step two. On retry, PAT recalls an alternative lesson, “sum,” and BRACE checks `1 + 1 = 2`. Step two now runs for the first time. It proposes multiplication but reports **5** for `2 × 2`. BRACE returns `wrong-result`. The retry is spent, so the final answer is withheld.\n\nThe example submits one candidate per step. The correct case spends two checks, each single-fault case spends four, and the two-fault case spends three. After NumPy and both repositories are available, the example runs offline. [Adapter](https://github.com/Onescale414/BRACE-Harness/blob/main/examples/pat/adapter.py) · [Runnable example](https://github.com/Onescale414/BRACE-Harness/blob/main/examples/pat/demo.py)\n\n- **Learning a domain from observations.** BRACE learned local action meanings and requested measurements to test them. With the same finite rule family and 38 readings, choosing inputs where rules disagreed yielded 114 correct fresh answers versus 55 from the next integer. Two errors remained when the true rule was absent, and selection required more internal computation.[Domain induction and transfer](https://github.com/Onescale414/BRACE-Harness/blob/main/research/domain-induction/README.md) ·[Missing-setting repair](https://github.com/Onescale414/BRACE-Harness/blob/main/research/domain-induction/missing-settings.md) ·[Choosing the next observation](https://github.com/Onescale414/BRACE-Harness/blob/main/research/domain-induction/disagreement-probes.md)\n- **Repeated additions.** A consecutive-integer detector flagged 22 of 24 fresh cases involving possible repeated additions and 6 of 8 outside controls. A later reader linked both amounts to their event text on 4 of 18 relevant requests, including a penalty that had been copied into a second record.[Repeat detection](https://github.com/Onescale414/BRACE-Harness/blob/main/research/arithmetic-handoff/README.md#does-a-repeated-integer-mean-a-repeated-operation) ·[Event attachment and replay](https://github.com/Onescale414/BRACE-Harness/blob/main/research/arithmetic-handoff/README.md#can-the-reader-attach-each-amount-to-its-event)\n- **Bracket geometry.** Two measured marks admit two possible tip positions, 96 drawing units apart. Reveal the third mark to distinguish them. Open the viewer locally or run`python -I -S -B preview/verify.py` to check the coordinates.[Bracket viewer](https://github.com/Onescale414/BRACE-Harness/blob/main/preview/bracket.html)\n- **Task handoff.** The experimental reader completed 25 of 36 connector requests. It still checked a URL that the user mentioned only as an example. The trial includes all wordings and an offline replay.[Delivery experiment](https://github.com/Onescale414/BRACE-Harness/blob/main/research/delivery/README.md)\n- **Which request did the caller mean?** When one API error fits three routes and a tool can only check two, it can ask which action the caller meant. Compare two readers on “this request,” “another request,” and replies that mention multiple actions. Run`python -I -S -B research/connector-clarification/comparison.py` .[Connector clarification example](https://github.com/Onescale414/BRACE-Harness/blob/main/research/connector-clarification/README.md)", "url": "https://wpnews.pro/news/bounded-checking-and-repair-for-ai-or-other-proposers", "canonical_source": "https://github.com/Onescale414/BRACE-Harness", "published_at": "2026-11-20 16:03:10+00:00", "updated_at": "2026-11-20 16:18:23.267570+00:00", "lang": "en", "topics": ["ai-safety", "ai-agents", "ai-tools", "developer-tools"], "entities": ["BRACE", "BRACE-Harness", "Python 3.12", "PAT", "MockServer", "WireMock"], "also_reported_by": [], "takeaway": "BRACE's connector domain checks which mapping was selected and binds it to the exact source and target before building the patch, because a correct conversion alone cannot establish that the right mapping was chosen.", "recommendation": null, "alternates": {"html": "https://wpnews.pro/news/bounded-checking-and-repair-for-ai-or-other-proposers", "markdown": "https://wpnews.pro/news/bounded-checking-and-repair-for-ai-or-other-proposers.md", "text": "https://wpnews.pro/news/bounded-checking-and-repair-for-ai-or-other-proposers.txt", "jsonld": "https://wpnews.pro/news/bounded-checking-and-repair-for-ai-or-other-proposers.jsonld"}}