{"slug": "torchlean-is-a-lean-4-framework-for-neural-networks", "title": "TorchLean is a Lean 4 framework for neural-networks", "summary": "Lean-dojo released TorchLean, an open-source Lean 4 framework that combines general tensor computation, machine learning and mathematical proofs in a single language, with automatic differentiation owned by TorchLean and GPU tensor values and local gradients computed through LibTorch's ATen operations. TorchLean reuses model definitions across training, graph transformations, certificate checks and proofs, and its verification tools can check properties such as whether a classifier keeps its prediction across an input region; through FloatLib, users select numerical precision and prove bounds on rounding and numerical error under explicit assumptions. The external GPU implementation remains a trust boundary separate from the Lean proofs, and the framework is installable as a Lake package via the lean-dojo/TorchLean GitHub repository, with quickstart commands running a two-layer MLP on CPU with FloatLib binary32 arithmetic or Lean's native Float32 and an optional CUDA build via TORCHLEAN_LIBTORCH_HOME.", "body_md": "TorchLean brings general tensor computations, machine learning, and mathematical proofs together in Lean. We can use its shape-checked tensors for linear algebra and numerical algorithms, write our own tensor functions, or build and train neural networks, from small examples to larger GPU-backed training runs. Lean is both a functional programming language and a theorem prover, so we can run a calculation and reason about it in the same language.\n\nWe reuse model definitions for training, graph transformations, certificate checks, and proofs. The verification tools cover questions such as whether a classifier keeps its prediction across an input region. Through FloatLib, we can choose the numerical precision and prove bounds on rounding and numerical error under explicit assumptions. TorchLean owns automatic differentiation; on GPU, it calls LibTorch's ATen operations to compute tensor values and local gradients. The external GPU implementation remains a trust boundary, separate from the Lean proofs.\n\n```\ngit clone https://github.com/lean-dojo/TorchLean.git\ncd TorchLean\nscripts/lake.sh exe cache get\nscripts/lake.sh build\n```\n\nFor Linux, macOS, Windows/WSL, CUDA with LibTorch, and an explanation of\nTorchLean's backend architecture, see the [Installation guide](https://lean-dojo.github.io/TorchLean/installation/).\n\n```\nscripts/lake.sh exe torchlean quickstart_mlp --device cpu --steps 10 --arithmetic ieee\nscripts/lake.sh exe torchlean quickstart_mlp --device cpu --steps 10\n\n# Optional GPU run with a CUDA-enabled LibTorch SDK, matching toolkit, and NVIDIA GPU:\nexport TORCHLEAN_LIBTORCH_HOME=/absolute/path/to/libtorch\nscripts/lake.sh -Kcuda=true build\nscripts/lake.sh -Kcuda=true exe torchlean quickstart_mlp --device cuda --steps 10\n```\n\nThe first quickstart uses [FloatLib](https://github.com/lean-dojo/FloatLib)'s binary32 arithmetic.\nThe second uses Lean's native `Float32`.\nTyped tensors and models also accept other FloatLib binary formats; see [Precision](#precision).\nThe CUDA command uses LibTorch's GPU runtime while TorchLean retains its differentiation tape.\nIt reports an error when CUDA is unavailable.\n\nApplication code writes concrete tensor types as `Tensor α [dims...]`, with the element type first.\nFor example, `Tensor Float [4, 2]` is a four-by-two tensor of `Float` values:\n\n``` python\nimport NN.API\nopen TorchLean\nopen Trainer.Objective (mse)\n\n/-- A two-layer regression model. The dimensions are checked when the layers are composed. -/\ndef model :=\n  nn.Sequential![\n    nn.linear 2 8,\n    nn.relu,\n    nn.linear 8 1\n  ]\n\n-- Four input rows, each containing two features.\ndef xs : Tensor Float [4, 2] :=\n  [[0.0, 0.0], [0.0, 1.0], [1.0, 0.0], [1.0, 1.0]]\n\n-- One regression target for each input row.\ndef ys : Tensor Float [4, 1] :=\n  [[0.2], [1.0], [1.0], [1.8]]\n\n-- The leading `4` counts samples; each sample has shapes `[2]` and `[1]`.\ndef data : Trainer.Dataset [2] [1] := Data.fromTensors xs ys\n\ndef trainOnce : IO Unit := do\n  -- Select the loss and train on the CPU with FloatLib binary32 arithmetic.\n  let trainer :=\n    Trainer.new model\n      { objective := mse\n        optimizer := optim.sgd { lr := 0.05 }\n        device := cpu\n        arithmetic := ieee }\n  -- Inspect the initialized model before any parameter updates.\n  let initialPrediction ← trainer.predict ([0.5, -0.25] : Tensor Float [2])\n  IO.println s!\"initial={reprStr initialPrediction}\"\n  -- Each step averages 16 sample gradients at one parameter point, then updates once.\n  -- Training returns a result that retains the updated parameters and run report.\n  let trained ← trainer.train data { steps := 200, samplesPerStep := 16, logEvery := 25 }\n  trained.printSummary\nscripts/lake.sh exe torchlean --help\nscripts/lake.sh exe verify --help\nscripts/lake.sh exe verify -- torchlean-ibp\n```\n\nFor the maintained examples:\n\n```\nscripts/lake.sh build NNExamples\n```\n\nTorchLean is a normal Lake package. You can depend on the Git repository directly:\n\n```\nrequire TorchLean from git \"https://github.com/lean-dojo/TorchLean.git\" @ \"main\"\n```\n\nThen run from the downstream project's root, using its own Lake configuration:\n\n```\nlake update\nlake exe cache get\nlake build\n```\n\nUse `import NN.API` for model, data, and training code. It provides `TorchLean.nn`,\n`TorchLean.Data`, `TorchLean.Trainer`, and `TorchLean.optim` without exposing the full proof and\nbackend trees. Use `NN.API.Verification` to call\n`trained.verify center (radius := r)` after ordinary training. Explicit verifier\ngraph lowering has the focused import `NN.API.Verification.Lowering`. Use `import NN` when the\nsame file also needs proofs or backend infrastructure; focused imports such as `NN.GraphSpec`,\n`NN.Runtime`, or `NN.Proofs` are available for subsystem work.\n\nWe can write a scalar calculation as an ordinary Lean function, apply it to a tensor, and choose the execution device at the call site:\n\n``` python\nimport NN.Kernel\nopen TorchLean\n\ndef square := fun (x : Float32) => x * x\n\ndef input : Tensor Float32 [3] := Tensor.ofFn fun i => Float32.ofNat (i.val + 1)\ndef onCpu : IO (Tensor Float32 [3]) := square.run input (device := cpu)\ndef onGpu : IO (Tensor Float32 [3]) := square.run input (device := gpu)\n\n#eval square.run input (device := cpu)\n-- [1.000000, 4.000000, 9.000000]\n```\n\nCPU runs the Lean function and is the default. GPU compiles supported calculations through NVRTC,\nusing native FP32/FP64 or configured binary arithmetic. A CPU-only scalar type or format retains\nits precision on CPU with a diagnostic. Unsupported code for a GPU-capable type, or an unavailable\nGPU, still returns an error. Functions can take runtime scalar parameters;\n`f.zip left right` combines two equally shaped tensors. Indexed programs support bounded loops and\nfixed-parameter numerical recurrences, with Lean-checked correspondence to the recursive equations.\nWhole-tensor functions use the same call: `f.run input`. Add `(grad := true)` to record supported\noperations on the existing autograd tape; the result has `.value` and `.backward`. Shapes and\nstorage are inferred from the input. Arbitrary indexed bodies are not differentiated automatically.\nSee the [custom-operation guide](https://github.com/lean-dojo/TorchLean/blob/main/NN/Kernel/README.md) for supported operations, indexed programs,\nlowering proofs, and the native execution boundary.\n\n[FloatLib](https://github.com/lean-dojo/FloatLib) supplies configurable arithmetic and numerical\nproofs. TorchLean's CPU tensors and typed models can use binary32, binary128, or a custom binary\nformat without converting through native floats. Custom GPU computations also accept configured\nbinary types, selecting compatible hardware operations automatically and using integer-limb\nsoftware arithmetic for custom formats and precisions beyond native floats.\nThe LibTorch tape retains binary32 for `Float32` and binary64 for `Float`, including gradients\nand optimizer state. Configured binary arithmetic also records on the GPU tape, retaining complete\nwords for saved values and gradients. This supports the arithmetic operations described in the\n[custom-operation guide](https://github.com/lean-dojo/TorchLean/blob/main/NN/Kernel/README.md#recording-a-calculation-for-differentiation);\nnative model operators and optimizer checkpoints still require native dtypes.\nDecimal and posit formats use CPU.\n\nSee the [typed-training example](https://github.com/lean-dojo/TorchLean/blob/main/NN/Examples/Quickstart/TypedTraining.lean) for binary128 training\nand the [tensor guide](https://lean-dojo.github.io/TorchLean/blueprint/Building-Models/Tensors-That-Remember-Their-Shapes/)\nfor precision, shapes, and derivatives. Import `NN.API.Precision` for the scalar integration, or\n`FloatLib` for standalone numerical work.\n\n- `NN.lean` : complete import for model, tensor, data, training, verification, and proof workflows.\n- `NN/API` : the application API exported by`import NN.API` and included by`import NN` .\n- `NN/Tensor` : the shared shape-indexed tensor type, packed CPU storage, conversions, and operations.\n- `NN/Spec` : mathematical tensor, layer, model, and dynamical-system definitions.\n- `NN/Runtime` : executable autograd, optimizers, training loops, CUDA boundary,\nPyTorch import/export, Jaxpr graph import, and RL runtime support.\n- `NN/Backend` : contract-carrying kernel capsules, the planner, backend profiles, execution\naudits, and the contract check that accepts or rejects a kernel plan.\n- `NN/IR` and`NN/GraphSpec` : graph semantics, per-node precision and explicit casts,\nand typed architecture descriptions.\n- `NN/Proofs` : tensor algebra, selected autograd correctness theorems, analytic derivatives,\nruntime approximation, and bridge proofs.\n- `NN/Floats` : TorchLean's scalar integration and numerical proof adapters.\n- FloatLib dependency: configurable executable formats, reference semantics, rounding proofs, and scalar intervals.\n- `NN/MLTheory` : learning theory, robustness, CROWN/LiRPA, generative objectives,\noptimization theory, and related proof layers.\n- `NN/Verification` : certificate checkers and CLI workflows.\n- `NN/Examples` : quickstarts, runnable model examples, widgets, bundled verification assets,\nand interoperability workflows.\n- `home_page/blueprint/TorchLeanBlueprint/Guide` : source for the guide.\n- `home_page` : project website sources.\n\nTorchLean proves properties of explicit Lean definitions. It also checks certificates produced by external tools, including bound-propagation and scientific-computing workflows. An executable certificate check reports acceptance by that checker. A semantic guarantee additionally requires the checker's soundness theorem and its hypotheses; a Lean proof needs kernel-checked evidence of acceptance. None of these checks certifies the program that produced the certificate.\n\nCPU instructions, CUDA kernels, cuBLAS, LibTorch, PyTorch, Julia, and other external systems are\nruntime providers. Their interfaces, assumptions, and available checks are listed in\n[`docs/TRUST_BOUNDARIES.md`](https://github.com/lean-dojo/TorchLean/blob/main/docs/TRUST_BOUNDARIES.md). Third-party sources and licenses are listed in\n[`docs/THIRD_PARTY_NOTICES.md`](https://github.com/lean-dojo/TorchLean/blob/main/docs/THIRD_PARTY_NOTICES.md), and\n[the AI usage disclosure](https://github.com/lean-dojo/TorchLean/blob/main/docs/CONTRIBUTING.md#ai-usage-disclosure) describes the\nproject's use of coding assistants.\n\nContribution guidelines are in [docs/CONTRIBUTING.md](https://github.com/lean-dojo/TorchLean/blob/main/docs/CONTRIBUTING.md).\n\nIf TorchLean is useful in your work, please cite\n[*TorchLean: Formalizing Neural Networks in Lean*](https://arxiv.org/abs/2602.22631):\n\n```\n@misc{george2026torchlean,\n  title         = {TorchLean: Formalizing Neural Networks in Lean},\n  author        = {George, Robert Joseph and Cruden, Jennifer and Adkisson, Will and\n                   Zhong, Xiangru and Zhang, Huan and Anandkumar, Anima},\n  year          = {2026},\n  eprint        = {2602.22631},\n  archivePrefix = {arXiv},\n  primaryClass  = {cs.MS},\n  url           = {https://arxiv.org/abs/2602.22631}\n}\n```\n\nTorchLean is released under the MIT License. See `LICENSE`.", "url": "https://wpnews.pro/news/torchlean-is-a-lean-4-framework-for-neural-networks", "canonical_source": "https://github.com/lean-dojo/TorchLean", "published_at": "2026-10-28 11:34:33+00:00", "updated_at": "2026-10-28 11:42:15.735319+00:00", "lang": "en", "topics": ["machine-learning", "ai-research", "neural-networks", "developer-tools", "ai-tools"], "entities": ["TorchLean", "Lean 4", "Lean", "LibTorch", "ATen", "FloatLib", "lean-dojo", "Lake"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/torchlean-is-a-lean-4-framework-for-neural-networks", "markdown": "https://wpnews.pro/news/torchlean-is-a-lean-4-framework-for-neural-networks.md", "text": "https://wpnews.pro/news/torchlean-is-a-lean-4-framework-for-neural-networks.txt", "jsonld": "https://wpnews.pro/news/torchlean-is-a-lean-4-framework-for-neural-networks.jsonld"}}