Neuro: a language that compiles training code to a native binary

I’ve spent about a year building Neuro, a programming language for AI work. You write the model, the training loop and the GPU code in one language, and the compiler produces a single native program. Whoever runs it doesn’t need Python installed.

This small program trains on the CPU, then predicts on the GPU:

@grad
func loss(w: &mut Tensor<f32, [2, 1]>) -> Tensor<f32, []> {
    val x: Tensor<f32, [4, 2]> = [[1.0, 2.0], [2.0, 0.0], [0.0, 1.0], [3.0, 1.0]]
    val y: Tensor<f32, [4, 1]> = [[5.0], [2.0], [2.0], [5.0]]
    val miss = x @ w - y
    return Tensor::scalar((&miss * &miss).mean())
}

@gpu
func predict(x: &Tensor<f32, [1, 2]>, w: &Tensor<f32, [2, 1]>) -> Tensor<f32, [1, 1]> {
    x @ w
}

func main() -> i32 {
    mut w: Tensor<f32, [2, 1]> = [[0.0], [0.0]]
    for step in 0..61 {
        val error = loss(&mut w)
        error.backward()
        w -= 0.05f32 * w.grad()
        w.zero_grad()
    }
    val x: Tensor<f32, [1, 2]> = [[1.0, 1.0]]
    val w_gpu = w.clone().to(Device::GPU(0))
    val y = predict(&x, &w_gpu).to(Device::CPU)
    println("predicted on the GPU: {y[0, 0]:.2}")
    return 0
}

The training loop should look familiar if you use PyTorch. Three things work differently.

Tensor shapes are part of the type, so a shape mismatch stops the build instead of crashing a run hours in. Swapping the product in loss to w @ x gives:

error: the operands of `@` do not multiply: &mut Tensor<f32, [2, 1]> against Tensor<f32, [4, 2]>

@grad generates the backward pass when you compile, so nothing records a graph while the program runs.

@gpu functions compile to NVIDIA or AMD kernels, on Linux only.

PyTorch is still faster on the GPU. Timed warm, it beats Neuro on all four GPU benchmarks, from about even on matmul to roughly 10x on reductions. Neuro has no shared-memory staging, and it stays off tensor cores because they would change the results in the last bits. A Neuro program does start faster, at about 220 ms against about 1100 ms for PyTorch’s import and CUDA setup. Full tables are in the README, and the benchmark script refuses to report timings if the implementations disagree on output.

It’s alpha, so please don’t expect to port a real model today. There’s no neural network library yet, so you write layers and optimizers by hand. Layers, attention, optimizers, a Data and .safetensors weights are all planned, which means you can’t load a model from the Hub yet. Multithreading and Python extension modules are missing too, and you have to build from source against LLVM and MLIR 23.

The license is shared source, not open source. You can use and modify the compiler, use it at work, and sell whatever you build with it. Shipping the compiler itself in a commercial product needs a commercial license or a renamed, source-available fork.

If you write training code, what would stop you from trying this? I’m designing the neural network library next, so that answer would change what I build.

Source: GitHub - PanzerPeter/Neuro: An AOT compiled programming language using LLVM · GitHub

Docs: https://neuro-lang.netlify.app