Neuro: a language that compiles training code to a native binary

Developer PanzerPeter released Neuro, an ahead-of-time compiled programming language that compiles model, training-loop and GPU code into a single native binary that runs without Python installed, after about a year of work. Neuro makes tensor shapes part of the type so mismatches fail at build time, generates backward passes at compile time via @grad, and compiles @gpu functions to NVIDIA or AMD kernels on Linux only, but PyTorch remains faster on the GPU — beating Neuro on all four GPU benchmarks, from roughly even on matmul to about 10x on reductions — while a Neuro program starts in about 220 ms versus about 1100 ms for PyTorch's import and CUDA setup. The alpha has no neural network library, multithreading or Python extension modules, requires building from source against LLVM and MLIR 23, and ships under a shared-source rather than open-source license.

I’ve spent about a year building Neuro, a programming language for AI work. You write the model, the training loop and the GPU code in one language, and the compiler produces a single native program. Whoever runs it doesn’t need Python installed. This small program trains on the CPU, then predicts on the GPU: php @grad func loss w: &mut Tensor<f32, 2, 1 - Tensor<f32, { val x: Tensor<f32, 4, 2 = 1.0, 2.0 , 2.0, 0.0 , 0.0, 1.0 , 3.0, 1.0 val y: Tensor<f32, 4, 1 = 5.0 , 2.0 , 2.0 , 5.0 val miss = x @ w - y return Tensor::scalar &miss &miss .mean } @gpu func predict x: &Tensor<f32, 1, 2 , w: &Tensor<f32, 2, 1 - Tensor<f32, 1, 1 { x @ w } func main - i32 { mut w: Tensor<f32, 2, 1 = 0.0 , 0.0 for step in 0..61 { val error = loss &mut w error.backward w -= 0.05f32 w.grad w.zero grad } val x: Tensor<f32, 1, 2 = 1.0, 1.0 val w gpu = w.clone .to Device::GPU 0 val y = predict &x, &w gpu .to Device::CPU println "predicted on the GPU: {y 0, 0 :.2}" return 0 } The training loop should look familiar if you use PyTorch. Three things work differently. Tensor shapes are part of the type, so a shape mismatch stops the build instead of crashing a run hours in. Swapping the product in loss to w @ x gives: error: the operands of @ do not multiply: &mut Tensor<f32, 2, 1 against Tensor<f32, 4, 2 @grad generates the backward pass when you compile, so nothing records a graph while the program runs. @gpu functions compile to NVIDIA or AMD kernels, on Linux only. PyTorch is still faster on the GPU. Timed warm, it beats Neuro on all four GPU benchmarks, from about even on matmul to roughly 10x on reductions. Neuro has no shared-memory staging, and it stays off tensor cores because they would change the results in the last bits. A Neuro program does start faster, at about 220 ms against about 1100 ms for PyTorch’s import and CUDA setup. Full tables are in the README, and the benchmark script refuses to report timings if the implementations disagree on output. It’s alpha, so please don’t expect to port a real model today. There’s no neural network library yet, so you write layers and optimizers by hand. Layers, attention, optimizers, a DataLoader and .safetensors weights are all planned, which means you can’t load a model from the Hub yet. Multithreading and Python extension modules are missing too, and you have to build from source against LLVM and MLIR 23. The license is shared source, not open source. You can use and modify the compiler, use it at work, and sell whatever you build with it. Shipping the compiler itself in a commercial product needs a commercial license or a renamed, source-available fork. If you write training code, what would stop you from trying this? I’m designing the neural network library next, so that answer would change what I build. Source: GitHub - PanzerPeter/Neuro: An AOT compiled programming language using LLVM · GitHub https://github.com/PanzerPeter/Neuro Docs: https://neuro-lang.netlify.app https://neuro-lang.netlify.app