{"slug": "rust-simd-on-the-gpu", "title": "Rust SIMD on the GPU", "summary": "VectorWare, a GPU-native software company, announced it can now use Rust's portable SIMD (core::simd) on the GPU, mapping a Simd<T, N> vector directly to a warp's 32 lanes. This milestone, a world first, completes the parallelism hierarchy by treating the GPU as vector hardware, enabling developers to write high-performance GPU applications using familiar Rust abstractions without needing std support.", "body_md": "[VectorWare](/)\n\nGPU code can now use Rust's portable SIMD. We share the implementation approach and what this unlocks for GPU programming.\n\nAt [VectorWare](/), we are building the first\n[GPU-native software company](/blog/announcing-vectorware/). Today, we are excited to\nannounce that we can successfully use Rust's portable SIMD\n([ core::simd](https://doc.rust-lang.org/core/simd/index.html)) on the GPU. This\nmilestone marks a significant step towards our vision of enabling developers to write\ncomplex, high-performance applications that leverage the full power of GPU hardware\nusing familiar Rust abstractions.\n\n## Parallelism below the thread\n\nWhen we [brought Rust threads to the GPU](/blog/threads-on-gpu/), we mapped each\n[ std::thread](https://doc.rust-lang.org/std/thread/) to a GPU\n\n[warp](https://modal.com/gpu-glossary/device-software/warp). This let us run many concurrent threads on the GPU but did not use the parallel\n\n[lanes](https://docs.nvidia.com/cuda/cuda-programming-guide/01-introduction/programming-model.html#warps-and-simt)within each thread/warp.\n\nOn the CPU, the abstraction for parallelism within a thread is\n[SIMD](https://en.wikipedia.org/wiki/Single_instruction,_multiple_data). A single instruction\noperates on several data elements packed into a vector unit: where scalar code adds\ntwo numbers, a SIMD add takes two vectors of, say, eight `f32`\n\nvalues and produces eight\nsums at once. This data parallelism is *inside* a single thread, below the level where the\noperating system schedules anything.\n\n## Rust's portable SIMD\n\nHistorically, writing SIMD in Rust meant reaching for the architecture-specific vendor\nintrinsics in [ core::arch](https://doc.rust-lang.org/core/arch/index.html), such as\n\n[on x86-64 or](https://doc.rust-lang.org/beta/core/arch/x86_64/fn._mm256_add_ps.html)\n\n`_mm256_add_ps`\n\n[on Arm. These intrinsics are specific to a single instruction set, so a program that runs on more than one architecture needs a separate implementation for each.](https://doc.rust-lang.org/beta/core/arch/arm/fn.vaddq_f32.html)\n\n`vaddq_f32`\n\nRust's [portable SIMD](https://doc.rust-lang.org/core/simd/index.html) instead adds a layer\nof abstraction above these\nintrinsics. It provides a single generic type\n[ Simd<T, N>](https://doc.rust-lang.org/core/simd/struct.Simd.html) that represents a\nvector of\n\n`N`\n\nelements of type `T`\n\n. A program writes its arithmetic, comparisons,\nreductions, and lane shuffles once against `Simd`\n\nand the compiler lowers them to whatever\nvector instructions the target CPU has.**At VectorWare, we realized the GPU is just one more piece of vector hardware for\nportable SIMD to target.** As a bonus, portable SIMD lives in `core`\n\nrather than `std`\n\nand it does not even need the [ std support we brought to the\nGPU](/blog/rust-std-on-gpu).\n\n## SIMT is SIMD\n\nGPUs execute in a model NVIDIA calls\n[SIMT](https://en.wikipedia.org/wiki/Single_instruction,_multiple_threads), or Single\nInstruction, Multiple Thread. A warp issues one instruction, and each of its 32 lanes runs\nthat instruction on its own data. One instruction operating on many data elements is *exactly*\nwhat SIMD means, and the per-lane addressing that SIMT adds does not change\nthat. A warp is a wide vector unit and a portable SIMD vector maps onto that unit directly.\n\nFor example, a `Simd<i16, 32>`\n\ngives one\n`i16`\n\nelement to each of the warp's 32 lanes, and adding two such vectors compiles to a single warp\ninstruction in which every lane adds its element at once.\n\nThis new mapping completes the parallelism hierarchy from our earlier work. On the CPU, a\nthread contains SIMD lanes, and on the GPU [our std::thread is a\nwarp](/blog/threads-on-gpu/) whose hardware lanes play the same role. In both cases,\n\n`core::simd`\n\ndrives those lanes.## A world first: `core::simd`\n\non the GPU\n\nAs with our earlier posts, this is hard to show visually because the code is ordinary\nRust. The same `core::simd`\n\ntypes that lower to x86-64 SIMD on a laptop lower to warp\noperations on the GPU, with no change to the source.\n\nHere we define a small portable SIMD routine and call it from `main`\n\n. It exercises\nthe core features of the model: elementwise arithmetic, a comparison that produces a\nlane mask, a `select`\n\ndriven by that mask, and a horizontal reduction across lanes.\n\nThe entry point is a normal `fn main`\n\nwith no GPU-specific annotations. Our toolchain\ncompiles it to a GPU kernel, and the result is printed from the device using our [ std\nsupport](/blog/rust-std-on-gpu).\n\nBelow is a recording of the program running on the GPU, producing the exact same output as\n[running it on the\nCPU](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2024&gist=c6fd3bb9bb99b2bb92b2255c3174ac7b).\n\n## Implementation\n\nAs previously mentioned, the mapping rests on a single observation: a warp is a vector\nunit whose lanes are individually addressable. Once `Simd<T, N>`\n\nis laid out\nper lane, each family of operations has a direct warp-level counterpart.\n\n**SIMD elementwise operations** are the easy case. Addition, multiplication, comparison, and\nthe other lane-wise operators come from ordinary Rust trait implementations on `Simd`\n\nsuch as\n[ Add](https://doc.rust-lang.org/std/simd/type.f32x32.html#impl-Add%3C%26Simd%3CT,+N%3E%3E-for-Simd%3CT,+N%3E). The GPU runs them natively.\n\n**SIMD reductions** such as\n[ reduce_sum](https://doc.rust-lang.org/core/simd/struct.Simd.html#method.reduce_sum)\nand\n\n[combine every lane into a scalar. These use the GPU's warp shuffle instructions to exchange and combine values across lanes, producing the same scalar result in every lane.](https://doc.rust-lang.org/core/simd/struct.Simd.html#method.reduce_max)\n\n`reduce_max`\n\n**SIMD cross-lane shuffles**, such as\n[ simd_swizzle!](https://doc.rust-lang.org/core/simd/macro.simd_swizzle.html) and\nrotates, move elements between lanes. Because a SIMD lane is a GPU warp lane, these map onto the\nsame warp shuffle primitives that make GPU lanes so good at exchanging data.\n\n**SIMD masks** map just as cleanly. A [ Mask<T, N>](https://doc.rust-lang.org/core/simd/struct.Mask.html) gives one predicate to each SIMD\nlane.\n\n[performs a selection in every warp lane. Horizontal mask queries such as](https://doc.rust-lang.org/core/simd/struct.Mask.html#method.select)\n\n`Mask::select`\n\n[and](https://doc.rust-lang.org/core/simd/struct.Mask.html#method.any)\n\n`any`\n\n[use GPU](https://doc.rust-lang.org/core/simd/struct.Mask.html#method.all)\n\n`all`\n\n[vote and ballot](https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#parallel-synchronization-and-communication-instructions-vote-sync)instructions.\n\nScalar values in the surrounding code, such as a loop counter or a constant, are computed\nidentically by every lane and so are simply replicated across the warp just like in ordinary\nCUDA. This is the same uniform-versus-varying distinction that data-parallel languages like\n[ISPC](https://ispc.github.io/) make explicit, except here it falls out of Rust's own types: a\nplain `f32`\n\nis uniform, a `Simd<f32, 32>`\n\nis varying.\n\n## Working with lanes\n\nThe one place the abstraction and the hardware do not line up is lane count.\nOn the CPU a `Simd<T, N>`\n\nallows any `N`\n\nfrom 1 through 64, but GPU hardware has a fixed\nwidth: 32 lanes on NVIDIA and 32 or 64 on AMD. The mapping is one to one only when `N`\n\nmatches that width. A smaller `N`\n\nleaves some lanes idle while a larger `N`\n\ngives some or\nall lanes more than one element to process.\n\nWhen there is more work than the warp is wide, we need a way to say which lanes do what. It helps to think of the warp as a small \"machine\" of its own: a fixed set of primitives for moving and combining data across lanes, plus invariants about which lanes are active and how much data each one holds. \"Programming\" it means placing work onto lanes within those rules.\n\n**At VectorWare, we give that machine an IR.** Rather than a standalone data structure, we encode\nit in Rust's type system using types, generics, const generics, and trait bounds. A program is composed of typed\noperations: ballots, shuffles, reductions, scans, gathers, scatters, atomics, and [strip\nmining](https://en.wikipedia.org/wiki/Loop_sectioning) for vectors wider than the warp.\nOperands, execution shape, and capacity are typed too. Because the operations carry their shape in the types, many invalid programs\ncannot be constructed at all.\n\nThe IR needs no interpreter on the GPU. Each operation lowers straight to the\ncorresponding instructions with zero cost over hand-written PTX. The same types let us run it on the CPU\ntoo. We built a reference interpreter that executes the IR deterministically, a kind of\n[Miri](https://github.com/rust-lang/miri) for warp-lane programming. We use it to\nsimulate GPU code and for [differential\ntesting](https://en.wikipedia.org/wiki/Differential_testing).\n\nOur work targets NVIDIA today, but nothing here is CUDA specific. AMD wavefronts and\nVulkan [subgroups](https://docs.vulkan.org/guide/latest/subgroups.html) expose similar\nprimitives and semantics. The IR itself is architecture-agnostic Rust.\n\n## Benefits\n\nThe same source runs on the CPU and the GPU. Code and libraries that already use portable SIMD become candidates for GPU execution without a rewrite.\n\nUnmodified CPU code can use GPU lane-level parallelism. GPU-aware code can still go further by using\n`core::arch`\n\nintrinsics that map directly to PTX.\n\nA `Simd<T, N>`\n\nis an ordinary owned value.\nThe borrow checker, lifetimes, and type checking apply to it exactly as they do on the\nCPU. We are not adding a GPU-specific vector type or a new set of annotations. We are\nmapping Rust's existing portable SIMD onto the GPU's native execution model. At\n[VectorWare](/), we are making GPUs behave like a normal Rust platform.\n\n## Downsides\n\nPortable SIMD is still unstable in Rust. It requires the nightly\n`#![feature(portable_simd)]`\n\n, and its surface may change before it\nstabilizes.\n\nVectors narrower than the warp leave lanes idle, and vectors wider than the warp turn each operation into more instructions. The abstraction is only zero cost when the vector width matches the number of warp lanes.\n\nNot every cross-lane operation maps to an efficient warp instruction. Shuffles that match\nthe hardware's supported patterns are cheap, but arbitrary permutations may need several\ninstructions or a trip through shared memory. Horizontal operations like reductions and\n`all`\n\n/`any`\n\nalso act as synchronization points within the warp, which constrains how\nfreely the scheduler can overlap work.\n\nWe had to change the compiler to make the abstraction sound when interacting with other Rust features. As this is uncharted territory, we are not yet confident we have covered every case.\n\n## Future work\n\nWith SIMD, [threads](/blog/threads-on-gpu), and [async](/blog/async-await-on-gpu) all\nmapped onto the GPU, the natural next step is composing\nthem: threads spreading work across warps, `core::simd`\n\nspreading data across the lanes\nwithin each warp, and async structuring the concurrency between them.\n\nWe are also interested in lowering matrix-shaped SIMD onto the GPU's [tensor\ncores](https://www.nvidia.com/en-us/data-center/tensor-cores/), and in auto-vectorizing\nordinary scalar Rust loops into `Simd`\n\noperations so that code gets warp-level\nparallelism without being written against `core::simd`\n\nat all. As [members of the Rust\ncompiler team](/team), we are keen to explore how much of this can happen in the compiler\nitself.\n\nA vector representation shared across the CPU and the GPU is valuable, though\nit is not clear that today's portable SIMD types are the right basis for one.\nFor one thing, they largely sit in a world of their own within the `core`\n\nand `std`\n\nAPIs. More exploration is necessary.\n\n## Is VectorWare only focused on Rust?\n\nThe speed at which we are able to make progress on the GPU is a testament to the power of Rust's abstractions and ecosystem.\n\nAs a company, we understand that not everyone uses Rust. Our future products will support multiple programming languages and runtimes. However, we believe Rust is uniquely well suited to building high-performance, reliable GPU-native applications and that is what we are most excited about.\n\n## Follow along\n\nFollow us on [X](https://x.com/vectorware),\n[Bluesky](https://bsky.app/profile/vectorware.com),\n[LinkedIn](https://www.linkedin.com/company/vectorware/), or subscribe to our\n[blog](/blog) to stay updated on our progress. We will be sharing more about our work in\nthe coming months. You can also reach us at [hello@vectorware.com](mailto:hello@vectorware.com).", "url": "https://wpnews.pro/news/rust-simd-on-the-gpu", "canonical_source": "https://www.vectorware.com/blog/simd-on-gpu/", "published_at": "2026-08-10 18:12:49+00:00", "updated_at": "2026-08-10 19:25:01.777478+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "developer-tools"], "entities": ["VectorWare", "Rust", "NVIDIA"], "alternates": {"html": "https://wpnews.pro/news/rust-simd-on-the-gpu", "markdown": "https://wpnews.pro/news/rust-simd-on-the-gpu.md", "text": "https://wpnews.pro/news/rust-simd-on-the-gpu.txt", "jsonld": "https://wpnews.pro/news/rust-simd-on-the-gpu.jsonld"}}