{"slug": "introducing-hierarchos-native-a-rust-vulkan-backend-for-transformer-training-and", "title": "Introducing Hierarchos-Native: A Rust + Vulkan Backend for Transformer Training and Inference on Consumer GPUs 💜", "summary": "An open-source developer released Hierarchos-Native, a Rust and Vulkan compute backend for transformer training, fine-tuning and inference that runs without Python, PyTorch, CUDA or ROCm at runtime. The project registers 146 text model architectures, including Llama, Qwen, DeepSeek, Gemma, Mistral and Falcon, and reports 14 modern transformer architecture families passing strict tiny-model training parity tests across two AdamW optimization steps with maximum absolute parameter divergence below 2e-7. The backend supports full-parameter AdamW training, LoRA/PEFT fine-tuning, SafeTensors loading, checkpointing and Hugging Face Hub integration, and targets consumer and integrated GPUs such as RDNA3 graphics in the ROG Ally and Intel HD Graphics 520; the developer states it is not yet faster than PyTorch or llama.cpp across the board.", "body_md": "Hey everyone! :3\n\nI’ve been working on a project called **Hierarchos-Native** for quite a while now, and I figured I’d finally share it here since a pretty big part of what I’m doing revolves around Hugging Face models and infrastructure!\n\nIt’s an open-source machine learning backend I’ve been building in Rust, using Vulkan compute for transformer training, fine-tuning, and inference.\n\nThe whole thing started because I’ve always been a huge nerd for low-level programming and squeezing performance out of hardware that probably has no business running the things I throw at it. I’ve spent a lot of time experimenting with older CPUs, integrated graphics, and consumer GPUs, and eventually started wondering how much of the traditional ML software stack I could replace with a completely native implementation.\n\nSo naturally, I decided to write my own backend. Because apparently I don’t know how to pick small projects. >w<\n\nThe idea behind Hierarchos-Native is to provide a cross-vendor ML runtime that doesn’t require Python, PyTorch, CUDA, or ROCm at runtime. Instead, it uses Rust for the host-side implementation and Vulkan compute shaders for GPU execution.\n\nAt this point, the project has grown quite a bit beyond what I originally imagined.\n\nThere are currently **146 registered text model architectures**, including families like Llama, Qwen, DeepSeek, Gemma, Mistral, and Falcon. That doesn’t mean all 146 are production-ready, though! I track architecture registration, execution support, and numerical validation separately because I really don’t want to misrepresent what the backend can do.\n\nIt also supports native training and inference paths for qualified architectures, full-parameter AdamW training, LoRA/PEFT fine-tuning, SafeTensors loading, checkpointing, and Hugging Face Hub integration for models, datasets, and tokenizers.\n\nOne of the parts I’m particularly proud of is the numerical validation work. I’ve been building reference comparisons to make sure the native Vulkan implementation behaves consistently with existing implementations rather than just producing outputs that look vaguely correct.\n\nThe current training parity results include 14 modern transformer architecture families passing strict tiny-model tests across two AdamW optimization steps, with maximum absolute parameter divergence below 2e-7.\n\nThose are small deterministic validation fixtures, not full production-model benchmarks, but getting everything to agree that closely has been pretty satisfying. :3\n\nAnother thing I’m really passionate about is **making ML workloads more practical on low-end and edge hardware**.\n\nI’ve been experimenting with GPUs like the RDNA3 graphics in my ROG Ally, along with much older hardware like Intel HD Graphics 520. There’s something incredibly satisfying about getting hardware that people tend to dismiss to actually do useful work.\n\nA lot of my development time has gone into memory management, GPU synchronization, compute shader implementations, and figuring out ways to minimize overhead without sacrificing correctness.\n\nI’m definitely not claiming Hierarchos-Native is faster than PyTorch or llama.cpp across the board. There’s still a ton of optimization and real-world compatibility work ahead of me, and I’m trying to approach that part carefully.\n\nBut I’m pretty happy with how far it’s gotten, especially considering how much of this has involved learning things the hard way and staring at Vulkan validation errors at completely unreasonable hours. x3\n\nI wanted to share it here because Hugging Face interoperability is a major part of the project, and I’d really love to hear from people who have experience working on transformer implementations, GPU runtimes, or alternative training frameworks.\n\nI’m particularly interested in what other developers think is missing from the current ecosystem when it comes to cross-vendor GPU acceleration and running ML workloads on resource-constrained hardware.\n\nIf anyone wants to poke through the implementation, try building it, suggest improvements, or just talk about Vulkan compute and ML infrastructure, I’d be super happy to chat!\n\nHere’s the repository:\n\nThanks for checking it out! I hope some of you find it interesting. :3", "url": "https://wpnews.pro/news/introducing-hierarchos-native-a-rust-vulkan-backend-for-transformer-training-and", "canonical_source": "https://discuss.huggingface.co/t/introducing-hierarchos-native-a-rust-vulkan-backend-for-transformer-training-and-inference-on-consumer-gpus/191439#post_1", "published_at": "2026-11-04 21:15:48+00:00", "updated_at": "2026-11-04 21:21:46.581933+00:00", "lang": "en", "topics": ["machine-learning", "ai-infrastructure", "ai-tools", "developer-tools", "large-language-models"], "entities": ["Hierarchos-Native", "Rust", "Vulkan", "Hugging Face", "Llama", "Qwen", "DeepSeek", "PyTorch"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/introducing-hierarchos-native-a-rust-vulkan-backend-for-transformer-training-and", "markdown": "https://wpnews.pro/news/introducing-hierarchos-native-a-rust-vulkan-backend-for-transformer-training-and.md", "text": "https://wpnews.pro/news/introducing-hierarchos-native-a-rust-vulkan-backend-for-transformer-training-and.txt", "jsonld": "https://wpnews.pro/news/introducing-hierarchos-native-a-rust-vulkan-backend-for-transformer-training-and.jsonld"}}