{"slug": "show-hn-tilery-vm-run-nvidia-cutile-gpu-kernels-on-a-cpu-no-gpu-required", "title": "Show HN: Tilery-VM – run Nvidia cuTile GPU kernels on a CPU, no GPU required", "summary": "Tilery-VM, a virtual machine for NVIDIA's CUDA Tile IR bytecode, enables developers to run cuTile GPU kernels on a CPU without requiring a GPU or CUDA setup. The open-source project, hosted on GitHub, parses TileIR MLIR or bytecode and interprets it on the CPU, supporting 89 op forms and 90 test cases with 69 expected values from real cuTile. This allows cuTile kernels written in Python or Rust to be developed and tested on any CPU, serving as a reference for other backends.", "body_md": "`tilery-vm`\n\nis a virtual machine for CUDA Tile IR bytecode.\n\nIt executes TileIR bytecode on a CPU, so cuTile kernels can be developed and tested without a GPU. TileIR is NVIDIA's open, language-agnostic IR for CUDA kernels (the PTX analogue for the tile programming model); cuTile is the user-facing language that emits it, from both the Python and Rust clients.\n\nno GPU, no CUDA, no setup step:\n\n```\nuv run examples/minimal.py   # [0.0320586  0.08714432 0.23688284 0.6439143 ]\ncargo test --workspace       # 136 tests\n```\n\nOne way you can use `tilery-vm`\n\nis by using tilery_vm as a cpu backend for `cuda-tile`\n\n.\n\nThis allows you to write native cutile in a environment that does not have a gpu, and run it on the cpu.\n\n```\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\n#     \"cuda-tile[tileiras]==1.4.0; sys_platform != 'darwin'\",\n#     \"cuda-tile==1.4.0; sys_platform == 'darwin'\",\n#     \"numpy\",\n#     \"tilery-vm\",\n# ]\n#\n# [tool.uv.sources]\n# tilery-vm = { path = \"../bindings\" }\n# cuda-tile = { path = \"../parity/osx-cutile/cutile-shim\", marker = \"sys_platform == 'darwin'\" }\n# ///\nimport tilery_vm.cpu  # noqa: F401\n\nimport cuda.tile as ct\nimport numpy as np\n\n@ct.kernel\ndef softmax(a, result):\n    x = ct.load(a, index=(0,), shape=(4,))\n    e = ct.exp(x - ct.max(x, axis=0))\n    ct.store(result, index=(0,), tile=e / ct.sum(e, axis=0))\n\na = np.array([1, 2, 3, 4], dtype=np.float32)\nresult = np.zeros(4, dtype=np.float32)\nct.launch(None, (1, 1, 1), softmax, (a, result))\n\nprint(result)  # [0.0320586  0.08714432 0.23688284 0.6439143 ]\n```\n\nor from rust\n\n```\nuse cutile::compile_api::KernelCompiler;\nuse tilery_vm::{Arg, launch};\n\n#[cutile::module]\nmod my_kernels {\n    use cutile::core::*;\n\n    /// out[i] = scalar + 1, for a length-`S` f32 tensor.\n    #[cutile::entry()]\n    fn add_scalar<const S: [i32; 1]>(output: &mut Tensor<f32, S>, scalar: f32) {\n        let scalar_tile: Tile<f32, S> = broadcast_scalar(scalar, output.shape());\n        let ones: Tile<f32, S> = broadcast_scalar(1.0f32, output.shape());\n        output.store(scalar_tile + ones);\n    }\n}\n\nfn main() {\n    // compile the DSL kernel to bytecode\n    let artifacts =\n        KernelCompiler::new(my_kernels::__module_ast_self, \"my_kernels\", \"add_scalar\")\n            .generics(vec![\"8\".into()]) // const S = [8]\n            .strides(&[(\"output\", &[1])])\n            .target(\"sm_89\")\n            .compile()\n            .expect(\"DSL compile failed\");\n    let bytecode = artifacts.bytecode().expect(\"bytecode serialization failed\");\n\n    // run it on the CPU - pass a tensor + a scalar, just like a launch\n    let mut output = vec![0.0_f32; 8];\n    launch(&bytecode, [1, 1, 1], &mut [Arg::tensor(&mut output), Arg::f32(5.0)])\n        .expect(\"run on CPU\");\n\n    println!(\"\\noutput = {output:?}\"); // scalar + 1 = [6, 6, 6, 6, 6, 6, 6, 6]\n}\n```\n\nthis repo contains a tileir mlir parser that converts the mlir into a executable module. the repo also contains a interpreter that can execute the module on the cpu. the interpreter is designed to be simple and easy to understand, and can be used as a reference for implementing other backends.\n\nwe also support processing tileir bytecode directly, which we first deserialize into a module, and then follow the existing interpreter path to execute the module on the cpu.\n\ntileir bytecode is the canonical representation of a compiled tile kernel, and is what the cuTile clients (Python and Rust) emit. on a GPU that bytecode is handed to NVIDIA's `tileiras`\n\nassembler, which lowers it to a cubin for a specific target. tilery-vm consumes the same bytecode and interprets it directly, which is what lets cuTile kernels run on any cpu.\n\ncorrectness is the whole point of this VM, so here is precisely what is verified today and what is not. ~89 op forms are implemented end-to-end (parse + execute).\n\n`crates/tilery-vm/optests/`\n\nholds **90 op cases** (`optests.mlir`\n\n+ `cases.json`\n\n,\nkept 1:1 by a test). **69 of them carry expected values captured from real cuTile\non an NVIDIA RTX 4090** (sm_89, cuda-tile 1.4.0) by `parity/cutile_capture.py`\n\nand\nstored in `optests/cutile_goldens.json`\n\n- all 69 match the VM. this is a stored\ncapture from one device configuration, not something re-run on every build.\n\nthe remaining 21 cases have no GPU golden. they are the memory/view/pointer/token\nsurface (`make_tensor_view`\n\n, `load_ptr_tko`\n\n, `reshape`\n\n, `permute`\n\n, `offset`\n\n, ...)\nand are checked against hand-written expectations only.\n\n`parity/bytecode_parity.py`\n\nruns **7 kernels** (`vadd`\n\n, `loopadd`\n\n, `row_sum`\n\n,\n`row_cumsum`\n\n, `gemm`\n\n, `softmax`\n\n, `atom`\n\n) through the *same* client-emitted\nbytecode on both a real GPU and the VM, comparing f32 outputs at `atol = rtol = 1e-4`\n\n. needs `cupy`\n\nand a real device.\n\n```\nuv run parity/bytecode_parity.py\ncargo test --workspace\n```\n\n136 tests: 97 parser tests (textual IR -> module), the 90-case optest table, plus interpreter, lexer and memory tests. no test in the rust suite touches a GPU.\n\n`--workspace`\n\nmatters: this workspace sets `default-members = [\"crates/tilery-vm\"]`\n\n,\nso a bare `cargo test`\n\nruns only 101 of them and skips the `tilery-parser`\n\nand\n`tilery-interpreter`\n\nunit tests.\n\n```\nparity/fetch_reference_corpus.sh\nuv run parity/op_parity.py\n```\n\nthis is *static* analysis - it diffs our parser's op dispatch against upstream's\n`Ops.td`\n\nand reports which mnemonics appear in our local `.mlir`\n\ncorpus. it does\nnot execute anything; executed evidence is the two sections above.\n\nthat is sound for MMA (16-bit in, f32 accumulate) but`f16`\n\n/`bf16`\n\n/`tf32`\n\nare accepted but represented as`f32`\n\n.*not*for elementwise chains, where hardware rounds at every step and the VM does not. raw sub-f32 buffer I/O is also wrong-width. there is no GPU parity evidence below f32 - everything above is f32/f64/int.`i16`\n\n, unsigned-as-distinct-types, and sub-byte floats (`f8E*`\n\n,`f4E*`\n\n) are not implemented; pointer buffers are rejected.- an unknown mnemonic surfaces as a generic parse error rather than a \"unsupported op\" diagnostic.\n- the interpreter is scalar, so a large forward pass takes seconds - this is a fidelity tool, not a fast runtime.\n- no CI yet; the parity scripts print results but do not gate on failure.\n\nApache-2.0 - see [LICENSE](/drbh/tilery-vm/blob/main/LICENSE).\n\nthis project is not affiliated with or endorsed by NVIDIA. it interoperates with\n`cuda-tile`\n\n(Apache-2.0) as a client; no NVIDIA source is vendored into this repo.\nthe macOS shim under `parity/osx-cutile/`\n\nbuilds against a wheel downloaded from\nPyPI at build time - see [ parity/osx-cutile/README.md](/drbh/tilery-vm/blob/main/parity/osx-cutile/README.md).", "url": "https://wpnews.pro/news/show-hn-tilery-vm-run-nvidia-cutile-gpu-kernels-on-a-cpu-no-gpu-required", "canonical_source": "https://github.com/drbh/tilery-vm", "published_at": "2026-07-30 15:14:36+00:00", "updated_at": "2026-07-30 15:22:15.092806+00:00", "lang": "en", "topics": ["artificial-intelligence", "developer-tools", "machine-learning", "ai-infrastructure"], "entities": ["NVIDIA", "Tilery-VM", "cuTile", "TileIR", "CUDA", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/show-hn-tilery-vm-run-nvidia-cutile-gpu-kernels-on-a-cpu-no-gpu-required", "markdown": "https://wpnews.pro/news/show-hn-tilery-vm-run-nvidia-cutile-gpu-kernels-on-a-cpu-no-gpu-required.md", "text": "https://wpnews.pro/news/show-hn-tilery-vm-run-nvidia-cutile-gpu-kernels-on-a-cpu-no-gpu-required.txt", "jsonld": "https://wpnews.pro/news/show-hn-tilery-vm-run-nvidia-cutile-gpu-kernels-on-a-cpu-no-gpu-required.jsonld"}}