{"slug": "cuda-oxide-eliding-bounds-checks-with-proof-carrying-views", "title": "cuda-oxide: eliding bounds checks with proof carrying views", "summary": "The cuda-oxide project introduces proof-carrying views that eliminate bounds-check overhead in CUDA kernels written in Rust, boosting GEMM performance from 2,942 to 7,159 GFLOPS (2.43x) with only ~0.1% safety cost and bit-identical results. The views encode buffer size contracts in the kernel interface, checked once on the CPU, and a new MIR pass enables the safe kernel to compile to the same PTX as the unsafe version.", "body_md": "`gemm`\n\nvs `gemm_views`\n\n```\ncargo oxide run gemm            # the baseline + its GFLOPS line\ncargo oxide run gemm_views      # correctness, contract rejection, bench table\ncargo oxide run gemm_views -- --verify-ptx\n```\n\nNumbers to remember: **2,942 -> 7,159 GFLOPS (2.43x), safety cost ~0.1%, results bit-identical.**\n\n- lets run run\n`cargo oxide run gemm`\n\n. Look at`Performance: 0.730 ms, 2942 GFLOPS`\n\n. - the inner loop in\n`gemm/src/main.rs`\n\n:`a[row*k+i] * b[i*n+col]`\n\n. Looks normal. It's safe Rust on a GPU, and most of that runtime is the safety tax.\n\n- Look at\n`gemm.ptx`\n\n, you’ll see the inner loop:- two\n`setp`\n\n+ guarded`bra`\n\nto`trap;`\n\n- blocks per\n`fma`\n\n. - Every\n`a[i]`\n\nis \"is`i`\n\ninside the slice? if not, stop the kernel\", per access, per thread.\n\n- two\n- The checks also split the loop into multiple basic blocks, which kills unrolling. Roughly 13 instructions per fma; ~5 are check overhead.\n- Why LLVM can't fix it:\n`a.len()`\n\nis an opaque parameter. Nothing in the IR connects`m * k`\n\nto it.*You*know the buffers are big enough; that fact lives in the host program. Everything that follows is about handing that fact to the compiler.\n\nLets see `sgemm_naive_views`\n\nnext to `sgemm_naive_raw`\n\nin `gemm_views`\n\n.\n\n- One slide of mental model: a matrix is one flat array,\n`stride`\n\n= row width, element`(row, col)`\n\nlives at`row * stride + col`\n\n. `a_mat.row(row, k)`\n\n:**one** check covers A's whole row.`b_mat.col(col, k)`\n\n:**one** check covers B's whole column (elements a full row width apart).`a_row.zip_exact(b_col)`\n\n:**one** length check; after that the dot loop's only compare is its own exit test. Same six instructions as the raw twin: load, load, fma, advance, compare, branch.- The C write:\n`tile_2d32_rt(coord, n)`\n\nwith a**1x1 tile**. \"Tile\" just means \"the rectangle this thread owns\"; naive GEMM's rectangle is one cell. One documented`unsafe`\n\nobligation: every thread passes the same row width.`n`\n\nis a kernel argument, so that holds by construction. - The contract line above the kernel:\n`requires = (k >= 1, a.len() >= m * k, b.len() >= k * n, c.len() >= m * n)`\n\n. Sizes are now part of the kernel's interface, checked once per launch on the CPU. Point at the run output: the deliberately oversized-K launch is rejected with the relation text, before the GPU does anything.\n\n- The\n`gemm_views`\n\nrun: matches CPU reference, safe vs raw**bit-identical**, then the bench table: 7,159 vs 7,161 GFLOPS. Safety costs ~0.1%. Against`gemm`\n\n's 2,942:**2.43x**. - Run\n`--verify-ptx`\n\n: the harness pins same branch counts, same load/store instruction multisets, zero traps, no leaked markers. The safe kernel doesn't just perform like the unsafe one; it compiles to the same machine code.\n\n`#[kernel(unchecked_indexing)]`\n\n: for shapes the views can't express. Deletes the indexing checks outright; the contract is`get_unchecked`\n\n's (out of bounds = UB). Only indexing; overflow and div-by-zero still trap. And it never leaks: a kernel calling an opted-in kernel's helper keeps its own checks (regression-tested at the PTX level).- Nugget if time allows: the safe loop initially refused to unroll. The iterator's\n`Option<(f32, f32)>`\n\nmerged as an*aggregate PHI*, which LLVM cannot split (SROA only handles memory). New mir-lower pass`scalarize_block_args`\n\nsplits those into scalar PHIs; that's the moment safe PTX became identical to raw.\n\n`requires`\n\n**is deliberately tiny:**- one comparison per relation,\n`.len()`\n\n, unsigned scalars,`+ - *`\n\n, literals. - being tiny makes it trustworthy: every name is validated against the kernel signature (typos = compile error) and all arithmetic is checked u64 (no wraparound false-passes).\n- Natural extensions: divisibility (\n`n % 16 == 0`\n\nwould delete grid-edge guards), alignment requirements, and ultimately feeding the proof to the device so even the per-thread view`Option`\n\nchecks disappear.\n\n- one comparison per relation,\n**Same trajectory for** kernel scalars are uniform across threads by construction, so a macro-minted`tile_2d32_rt`\n\n's one`unsafe`\n\n:`Uniform<u32>`\n\nwitness type would make it fully safe.**A limitation:** partial edge tiles. Tile types have a fixed shape; if the rectangle doesn't fully fit (a 100x100 matrix with 16x16 tiles leaves 4-wide strips at the edges), the constructor returns`None`\n\nrather than a clipped tile. Today: pick sizes that divide, use 1x1 tiles (naive GEMM sidesteps this entirely), or guard edges per element. A clipped-tile type is future work.**Close on the ladder:** check every access -> check once per thread -> check once per launch. Each rung is the same check, done less often, and the type system is what lets you climb down without losing safety.", "url": "https://wpnews.pro/news/cuda-oxide-eliding-bounds-checks-with-proof-carrying-views", "canonical_source": "https://gist.github.com/nihalpasham/84ccb7d1cdd3ca6bcf2ed9a71e7c5c12", "published_at": "2026-08-02 03:28:09+00:00", "updated_at": "2026-08-02 11:30:21.643265+00:00", "lang": "en", "topics": ["developer-tools", "machine-learning", "artificial-intelligence"], "entities": ["cuda-oxide", "LLVM", "Rust", "CUDA", "GEMM"], "alternates": {"html": "https://wpnews.pro/news/cuda-oxide-eliding-bounds-checks-with-proof-carrying-views", "markdown": "https://wpnews.pro/news/cuda-oxide-eliding-bounds-checks-with-proof-carrying-views.md", "text": "https://wpnews.pro/news/cuda-oxide-eliding-bounds-checks-with-proof-carrying-views.txt", "jsonld": "https://wpnews.pro/news/cuda-oxide-eliding-bounds-checks-with-proof-carrying-views.jsonld"}}