{"slug": "llmll-a-goto-fail-body-rejected-before-the-edit-lands", "title": "Llmll: A goto-fail body, rejected before the edit lands", "summary": "LLMLL, a programming language designed for AI agents to write, rejects an agent-proposed function body that reproduces Apple's 2014 goto-fail TLS bug (CVE-2014-1266) before the edit lands, according to a 29 October 2026 demonstration by Burc Sahinoglu. The compiler hands each proposed body and its contract to an SMT solver; the wrong fill returning Verified on every path failed with an lh-unsafe diagnostic stating the else-branch does not satisfy the postcondition, leaving the file byte-for-byte unchanged, while the guarded fill returning (Rejected c) on Abort passed with body_faithful true and a SAFE result from liquid-fixpoint. The demonstration shows the rejection comes from the verifier rather than the type checker, since llmll build compiles the wrong body without complaint.", "body_md": "# LLMLL: a goto-fail body, rejected before the edit lands\n\n*Burc Sahinoglu · 29 October 2026. A self-contained demonstration. The [five-post\nseries](index.md) tells the longer story, up to 163 functions.*\n\nLLMLL is a programming language built for AI agents to write. A function starts as a contract, a precondition and a postcondition, with a typed hole where the body goes. An agent checks the hole out, proposes a body, and the compiler hands the body and the contract to an SMT solver. A body the solver cannot prove is rejected before it lands, and the program is left as it was.\n\nThis page runs that loop once, on the invariant behind a real bug, and shows every diagnostic it produces.\n\nThe two fills in the recording are scripted stand-ins for agents: fixed patch files committed to\nthe repository, so the rejection is reproducible. Script:\n[`examples/gotofail/demo.sh`](https://github.com/machunter/llmll/blob/main/examples/gotofail/demo.sh).\n\n## The contract\n\nIn 2014, Apple’s goto-fail bug (CVE-2014-1266) reported a TLS handshake as successful on a path\nwhere the signature check never ran. The mistake was about whether a check executes. Written as\na contract, the invariant is: return `Verified` only if the check passed.\n\n```\n(type Step    (| Continue) (| Abort int))\n(type Verdict (| Verified) (| Rejected int))\n\n(def-shell finalize [sig: Step payload: int] -> Verdict\n  (post (=> (= result Verified) (= sig Continue)))   ;; return Verified => sig was Continue\n  ?body)\n```\n\n`?body` is the hole. An agent asks for it with `llmll checkout`, which locks the hole and returns\na brief: the postcondition, the return type, the names in scope and a lock token. The brief has\nno reference body and no hint.\n\n``` js\n{\n  \"postcondition_goal\": \"(=> (= result Verified) (= sig Continue))\",\n  \"expected_return_type\": \"Verdict\"\n}\n```\n\n## A wrong fill is rejected\n\nSay the agent makes the goto-fail mistake and returns `Verified` on every path:\n\n```\n(match sig ((Continue) Verified) ((Abort c) Verified))\n```\n\nIt submits the body as a JSON-Patch against the program’s JSON-AST, with `--require-proof`:\n\n``` bash\n$ llmll patch finalize-hole.ast.json wrong.json --require-proof\n{\n  \"diagnostics\": [\n    {\n      \"kind\": \"lh-unsafe\",\n      \"message\": \"body verification of 'finalize' failed (else-branch does not satisfy postcondition) (constraint #1)\",\n      \"pointer\": \"patch-op/1\",\n      \"severity\": \"error\"\n    }\n  ],\n  \"result\": \"PatchVerifyError\"\n}\n```\n\nThe patch is refused and the file is byte-for-byte unchanged. The diagnostic names the branch\nthat fails: the `else` branch, the one that returns `Verified` when `sig` is `Abort`. That is\nthe input the real bug shipped on.\n\n## The guarded fill is accepted\n\n```\n(match sig ((Continue) Verified) ((Abort c) (Rejected c)))\nbash\n$ llmll patch finalize-hole.ast.json right.json --require-proof\n{\"result\":\"PatchSuccess\",\"reuse_suggestions\":[],\"statements\":3,\"verification\":[{\"body_faithful\":true,\"fn\":\"finalize\"}]}\n\n$ llmll verify finalize-hole.ast.json\n   body-faithful: finalize\n   Running liquid-fixpoint ...\n✅ finalize-hole.ast.json — SAFE (liquid-fixpoint)\n```\n\n`body_faithful` means the solver proved the postcondition from the body itself. It did not\nassume the contract.\n\n## What the contract leaves open\n\nThis contract is a safety property. It forbids returning `Verified` when the check failed,\nand it does not require returning `Verified` when the check passed, so a body that always\nreturns `(Rejected 0)` also verifies. Ruling that out takes a stronger contract, with `<=>`\nin place of `=>`. [Post 2](https://machunter.github.io/llmll/blog/post-2-a-compiler-that-refuses.html#what-the-contract-does-not-say)\nruns both versions.\n\n## Which pass rejects it\n\nThe rejection comes from the verifier, not from the type checker. `llmll build` checks types and\nemits Haskell, and it compiles the wrong body without complaint. The proof runs in ```\nllmll\nverify\n```\n, and in `llmll patch`, which re-verifies the patched function before writing anything.\nWith `--require-proof`, a patch is also refused when its function passes only because the\nsolver fell back to assuming the postcondition.\n\n## What this shows, and what it does not\n\n**Why a language for this, and not a verifier behind an agent tool.** The verifier is not the\nnew part: Dafny, Liquid Haskell or F* would reject the same body, and LLMLL’s proofs run on\nliquid-fixpoint and Z3. What LLMLL adds is the loop around the proof, built into the\ntoolchain:\n\n- The unit of work is a hole with a contract. `checkout` hands an agent that hole’s contract\nand the names in scope as JSON, and locks it.\n- `patch` applies the agent’s body and verifies it before writing, so a rejected edit leaves\nthe program byte-for-byte unchanged, and the lock stays with the agent so it can retry.\n- A rejection is JSON with a kind, a message and a pointer into the patch, so a repair loop can act on it without parsing compiler text.\n- `refine` lets an agent split a hole into helper holes with contracts of their own, while\nthe parent stays held to its original contract\n([Post 4](https://machunter.github.io/llmll/blog/post-4-who-writes-the-decomposition.html) ).\n\nNone of this needs a stronger solver. It lets you hand one function at a time to an agent\nand take back only bodies that verify, with the rest of the program untouched. How contracts\ncompose across calls is [Post 3](https://machunter.github.io/llmll/blog/post-3-composition-and-the-missing-bound.html).\n\n**Scope.**\n\n- As of `llmll` 0.33, the solver covers linear integer arithmetic, booleans, non-recursive\nsums, pairs, fixed-size bytes, maps and string literals, each with restrictions listed in[the language reference](https://github.com/machunter/llmll/blob/main/LLMLL.md#533-verification-scope) .\nOutside that fragment a postcondition is assumed,`verify` names the function, and`--require-proof` refuses the fill.\n- A proof shows that the body meets the contract, for every input that satisfies the precondition. It does not show that the contract is right. Whether the contract says the right thing is a human reviewer’s job.\n- The example is one function. The series scales it up to an arithmetic model of the TLS record\nlayer’s control flow and bounds: 163 agent-written functions, verified as one program\n([Post 5](https://machunter.github.io/llmll/blog/post-5-the-payoff.html) ). That model computes no hash, MAC or cipher, and it is\nnot a verified TLS implementation.\n- A verdict trusts its tools: LLMLL’s translation to solver constraints, liquid-fixpoint, Z3, the Haskell code generator and GHC.\n\n## Run it\n\nWith Docker and nothing else, see the rejection:\n\n```\ndocker run --rm ghcr.io/machunter/llmll verify /opt/llmll/examples/gotofail/finalize-bad.llmll\n```\n\nIt prints ```\nerror: body verification of 'finalize' failed (else-branch does not satisfy\npostcondition)\n```\n. Swap `finalize-bad` for `finalize` to see the guarded body verify.\n\nTo run the whole checkout-and-patch loop from the recording, build `llmll` from\n[the repository](https://github.com/machunter/llmll) and run\n`bash examples/gotofail/demo.sh`. It needs `liquid-fixpoint`, `z3` and `jq` on your PATH.\n\n## Try to break it\n\nTwo kinds of report are useful, and they show different things:\n\n- **A body that breaks its contract and still verifies.** That is a verifier bug, and the\nmost serious kind of report this project can get.\n- **A body that meets its contract and is still wrong** , like the always-reject body above.\nThat shows a contract that says too little.\n\nEither way, [open an issue](https://github.com/machunter/llmll/issues).\n\n## Read on\n\nThe [series](https://machunter.github.io/llmll/) goes from this one function to 163:", "url": "https://wpnews.pro/news/llmll-a-goto-fail-body-rejected-before-the-edit-lands", "canonical_source": "https://machunter.github.io/llmll/demo.html", "published_at": "2026-10-30 19:37:41+00:00", "updated_at": "2026-10-30 19:46:41.794131+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "ai-safety"], "entities": ["LLMLL", "Burc Sahinoglu", "Apple", "CVE-2014-1266", "liquid-fixpoint"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/llmll-a-goto-fail-body-rejected-before-the-edit-lands", "markdown": "https://wpnews.pro/news/llmll-a-goto-fail-body-rejected-before-the-edit-lands.md", "text": "https://wpnews.pro/news/llmll-a-goto-fail-body-rejected-before-the-edit-lands.txt", "jsonld": "https://wpnews.pro/news/llmll-a-goto-fail-body-rejected-before-the-edit-lands.jsonld"}}