{"slug": "fearless-concurrency-on-the-gpu", "title": "Fearless Concurrency on the GPU", "summary": "Researchers introduced cuTile Rust, a tile-based system for safe, idiomatic GPU kernel authoring in Rust that extends Rust's ownership discipline to GPU kernels. On the NVIDIA B200 GPU, cuTile Rust achieved 7 TB/s for element-wise operations and 2 PFlop/s for GEMM (96% of cuBLAS), while Grout, a cuTile-Rust-based inference engine, reached 171 tokens/s for Qwen3-4B on the RTX 5090 and 82 tokens/s for Qwen3-32B on the B200, competitive with vLLM and SGLang.", "body_md": "# Computer Science > Programming Languages\n\n[Submitted on 14 Jun 2026]\n\n# Title:Fearless Concurrency on the GPU\n\n[View PDF](/pdf/2606.15991)\n\n[HTML (experimental)](https://arxiv.org/html/2606.15991v1)\n\nAbstract:Rust has made safe systems programming practical on the CPU, but writing custom GPU kernels in Rust still forces programmers outside the language's ownership guarantees. We present cuTile Rust, a tile-based system for safe, idiomatic GPU kernel authoring in Rust. cuTile Rust extends Rust's ownership discipline to tile-based GPU kernels: mutable outputs are split into disjoint pieces, kernel launches preserve the host-side ownership contract, and programmers can opt out locally when they need lower-level control. The system also provides a composable host execution model spanning synchronous launches, asynchronous pipelines, and CUDA graph replay.\n\nOur evaluation shows that these abstractions can preserve performance on high-end GPUs. On the NVIDIA B200 GPU, cuTile Rust achieves 7 TB/s for element-wise operations and 2 PFlop/s for GEMM (96% of cuBLAS), matching cuTile Python within measurement noise. Grout, a cuTile-Rust-based inference engine, exercises cuTile Rust across an end-to-end Qwen3 inference path. In batch-1 decode, Grout reaches 171 generated tokens/s for Qwen3-4B on the NVIDIA GeForce RTX 5090 and 82 generated tokens/s for Qwen3-32B on the B200, competitive with vLLM and SGLang and consistent with an HBM roofline sanity check.\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/fearless-concurrency-on-the-gpu", "canonical_source": "https://arxiv.org/abs/2606.15991", "published_at": "2026-06-17 06:33:59+00:00", "updated_at": "2026-06-17 06:52:47.333866+00:00", "lang": "en", "topics": ["machine-learning", "ai-infrastructure"], "entities": ["cuTile Rust", "NVIDIA B200", "cuBLAS", "Grout", "Qwen3-4B", "Qwen3-32B", "NVIDIA GeForce RTX 5090", "vLLM"], "alternates": {"html": "https://wpnews.pro/news/fearless-concurrency-on-the-gpu", "markdown": "https://wpnews.pro/news/fearless-concurrency-on-the-gpu.md", "text": "https://wpnews.pro/news/fearless-concurrency-on-the-gpu.txt", "jsonld": "https://wpnews.pro/news/fearless-concurrency-on-the-gpu.jsonld"}}