{"slug": "tla-verus-llm-agents-prove-temporal-properties-of-tla-specs", "title": "TLA-Verus: LLM agents prove temporal properties of TLA+ specs", "summary": "A NeurIPS'26 workshop paper, TLA-Verus, translates TLA+ specifications from GitHub into Verus and uses LLM agents to prove their safety and liveness properties, building on the temporal-logic library from Anvil (OSDI'24). The work targets TLA+'s weak proof support, though it leaves open the gap between spec and code and provides no refinement proof that the translated Verus spec preserves the original's meaning. A reviewer notes the paper omits comparison against TLAPS temporal proofs, which the critique argues are stronger than the paper suggests.", "body_md": "[This NeurIPS'26 workshop paper](https://arxiv.org/pdf/2610.14923) takes TLA+ specs from GitHub, translates them into Verus, and has LLM agents prove the safety and liveness properties in the specs.\n\nThe paper aims to address two problems it raises with TLA+. First, nothing ties the spec to the code, so they drift apart. And second, it claims that proof support is thin: TLC only checks small configurations, and TLAPS is weak on liveness.\n\nThey abandon pursuing the first weakness they raise at the onset of the paper. Closing it needs a proof that the generated Rust code (which it turns out TLA-Verus doesn't produce anyway) refines the spec, so they list this as future work. It gets worse though. There is no refinement proof between the TLA+ spec and the Verus spec either. Nothing establishes that the translated Verus spec means what the original meant. (Trace validation, meanwhile, is a viable way of tying TLA+ to implementation code, [and has been getting a lot of attention recently.](https://muratbuffalo.blogspot.com/2026/08/specula-scaling-formal-specifications.html))\n\nThey dedicate the paper to addressing the second weakness, but I think they  understate the TLAPS capability for doing temporal proofs. Writing `BY PTL` on a proof step tells TLAPS to treat that step as propositional temporal logic and send it to LS4, an LTL decision procedure. [Coalescing (Doligez, Kriener, Lamport, Libal, Merz)](https://arxiv.org/abs/1409.3819) is the technique that abstracts the first-order parts of a TLA+ obligation into propositional atoms LS4 can handle. Lamport's temporal proof rules are all there, including WF1, the rule that turns a weak fairness assumption on an action into a leads-to conclusion. The limitation is that TLAPS reasons poorly about ENABLED, and discharging the enabledness side conditions of WF1 is where a liveness proof needs more work. All this to say, TLAPS has temporal proof capability, and last time I used Claude Code to write TLAPS proofs, I was surprised by how well Opus was able to write TLAPS proofs. I expected the paper to compare against TLAPS proofs, but unfortunately that is missing.\n\n[Verus, the Rust verifier,](https://github.com/verus-lang/verus) is really cool though. It can host temporal logic. It is auto-active, which is the middle ground between a model checker you push a button on and a proof assistant like Coq or Lean you drive yourself. Here you annotate the program with preconditions, invariants, and lemmas, and an SMT solver discharges the resulting obligations automatically. So instead of writing a proof, your job (or more like your LLM's job) is to figure out which hint the solver is missing. [Anvil (OSDI'24)](https://www.usenix.org/conference/osdi24/presentation/sun-xudong) demonstrated this capability by proving liveness of Kubernetes controllers, and it did so by building temporal logic as a Verus library, since Verus itself knows nothing about [] and <>. This paper takes that library almost directly to use `always`, `eventually`, `leads_to`, weak fairness, and pre-proven lemmas like WF1. The game plan of the paper is to move the TLA+ spec into Verus, prove things there leveraging Anvil, and later (in the long term plan) prove that the Rust code written refines the proven Verus `spec`.\n\nSome background on Verus is in order here. A Verus file holds three kinds of code. `spec` code is mathematics: it never runs, and it can use unbounded integers and infinite sets. `proof` code is ghost code, lemmas that exist during verification and are erased afterward. `exec` code is real Rust that compiles and ships, and its preconditions and postconditions are written in terms of the `spec` functions. All three live in one file to keep the specification from drifting away from the implementation.\n\nIn this paper there is no `exec` code. Every task is `spec` and `proof` only. The translated TLA+ state machine is a `spec` function relating one state to the next, and what the LLM fills in is an empty `proof fn`. Nothing here compiles to a binary executable code. So the distinguishing feature of Verus from other SMT-backed provers is turned off for the entire corpus. Frankly, I can't see how code generation for a **distributed system** could be made from `spec` automatically even with LLMs.\n\nOK, I got carried away with the motivation and went into too much detail already. Let's backup and checkout the pipeline. The paper's main contribution is the prover-reviewer loop, which is a way to get LLMs to prove temporal properties of TLA+ specs. The pipeline has three stages: translation, proof search, and merge control.\n\nThe unit of work throughout is a (spec, property) pair: that is a whole TLA+ specification together with a single property to prove about it. Each pair becomes its own self-contained Verus file carrying exactly one open proof obligation. (Me thinks this is not a sustainable way to generate code from a real system.) Since a module usually declares several properties, pairs provide an inflated way to count: the 16,459 pairs come from only 2,734 TLA+ modules in 838 GitHub repositories, a median of 5 properties per module. The same specification gets translated and proved over and over, once per property.\n\nTranslation comes in two flavors. TranSpec is a deterministic Rust transpiler. It infers types for untyped TLA+, synthesizes termination measures, and places SMT triggers. The other flavor is an LLM translator checked by an LLM judge, without any proof for the translation.\n\nProof search uses a prover agent and a reviewer agent with fresh context. Verus only says which assertion failed, but gives no counterexample, so the reviewer agent supplies the semantic feedback.\n\nA gatekeeper controls the merges, a signed append-only ledger, and a git hook. Anti-cheat checks reject any net-new `assume`, `admit`, or `external_body`. The prover cannot merge its own work. This prevents the proof agent from reward hacking.\n\nThe paper runs three separate experiments. The first is the full corpus sweep via the prover-reviewer pipeline, driven by Opus. They tried 16,459 (spec, property) pairs. TranSpec translated 6,070 and 5,160 were well-posed. The prover closed 3,094 of those. Liveness was the weakest type: 304 of 809, about 38%. Proofs cost about $2 each. On the agentic side the losses move to a different stage: 2,851 pairs are judged unprovable by an LLM and dropped, with nothing checking that verdict.\n\nThe surviving corpus is lopsided. The paper splits safety into three buckets. \"Invariance\" is a plain state invariant: []P for a state predicate P. \"TypeOK\" is the type-correctness invariant, pulled out separately. \"Safety\" category is reserved for everything that is not a single state predicate. Of the 5,160 well-posed algorithmic tasks, invariance and TypeOK together are 83%, liveness is 809 tasks or 16%, and that last \"safety\" bucket is 50 tasks, 1%. Safety is scarce because TranSpec is bad at those forms rather than because GitHub lacks them: of the 630 safety properties the agentic pipeline translated, 578 have no usable algorithmic bundle at all.\n\nThe third experiment drops the reviewer and the gatekeeper. A model gets the task file and runs Verus, up to 10 attempts. So the prover-reviewer loop, the paper's most interesting contribution, is absent from every number in the model table.\n\nThe set is 40 tasks, 10 of each property type, in both encodings, and eleven models at pass@3. Opus-5 and GPT-5.6-sol-pro solve everything; gpt-oss-120b solves nothing. Nine of eleven do worse than on verusbench, DeepSeek-R1 falling from 74.7% to 17.5%. Liveness is again the hardest type: models solved 58 of 110 liveness attempts, against 73 to 78 for each of the other three types.\n\nThat 53% on liveness looks better than the 38% from the corpus sweep, but the two are not the same question. The second of seven selection criteria is that the task be \"known provable\", so the 40 tasks are drawn from the 3,094 the prover already closed. The ten liveness tasks come out of the 304 successes, never the 809 attempts. Which is also why the top of the table saturates: Opus-5 scores 100% on a set filtered by what Opus-5 could prove.\n\nI think the task generator is really the useful thing here. Normally a person has to construct/curate a usable/challenging task, which is why verusbench has only 150 tasks. Here Verus decides it. In TLA-Verus, a translation is kept only if it compiles and verifies with exactly one error, the open proof obligation. Zero errors means the property was already true with an empty proof, so there is nothing to prove (145 such tasks were dropped). Two or more errors mean the translation is broken. That said, this check also has drawbacks. Compiling with one error says the file is well-formed Verus with a proof obligation pending, but it does not guarantee that a proof is possible. The property may be false in the original spec, or the translation may have made it false, and then the prover just burns tokens only to record this task as unproven. The paper admits this type of impossible proof is indistinguishable from a proof that was merely hard. The opposite case is worse still: if the translation breaks the state machine instead of the property, the property holds for free and the proof is easy. That is shown in Appendix C.2: their vacuity filter misses it, because that filter only catches tasks that verify with an empty proof. But this may not be a big problem, because these specs come from GitHub and most were model-checked by their authors before being committed, so the properties would be \"mostly\" true.", "url": "https://wpnews.pro/news/tla-verus-llm-agents-prove-temporal-properties-of-tla-specs", "canonical_source": "https://muratbuffalo.blogspot.com/2026/10/tla-verus-llm-agents-prove-temporal.html", "published_at": "2026-10-27 19:13:12+00:00", "updated_at": "2026-10-27 19:42:34.898735+00:00", "lang": "en", "topics": ["ai-agents", "ai-research", "large-language-models", "artificial-intelligence"], "entities": ["Verus", "TLA+", "TLAPS", "Anvil", "NeurIPS", "Claude Code", "Leslie Lamport"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/tla-verus-llm-agents-prove-temporal-properties-of-tla-specs", "markdown": "https://wpnews.pro/news/tla-verus-llm-agents-prove-temporal-properties-of-tla-specs.md", "text": "https://wpnews.pro/news/tla-verus-llm-agents-prove-temporal-properties-of-tla-specs.txt", "jsonld": "https://wpnews.pro/news/tla-verus-llm-agents-prove-temporal-properties-of-tla-specs.jsonld"}}