Hey everyone! :3
I’ve been working on a project called Hierarchos-Native for quite a while now, and I figured I’d finally share it here since a pretty big part of what I’m doing revolves around Hugging Face models and infrastructure!
It’s an open-source machine learning backend I’ve been building in Rust, using Vulkan compute for transformer training, fine-tuning, and inference.
The whole thing started because I’ve always been a huge nerd for low-level programming and squeezing performance out of hardware that probably has no business running the things I throw at it. I’ve spent a lot of time experimenting with older CPUs, integrated graphics, and consumer GPUs, and eventually started wondering how much of the traditional ML software stack I could replace with a completely native implementation.
So naturally, I decided to write my own backend. Because apparently I don’t know how to pick small projects. >w<
The idea behind Hierarchos-Native is to provide a cross-vendor ML runtime that doesn’t require Python, PyTorch, CUDA, or ROCm at runtime. Instead, it uses Rust for the host-side implementation and Vulkan compute shaders for GPU execution.
At this point, the project has grown quite a bit beyond what I originally imagined.
There are currently 146 registered text model architectures, including families like Llama, Qwen, DeepSeek, Gemma, Mistral, and Falcon. That doesn’t mean all 146 are production-ready, though! I track architecture registration, execution support, and numerical validation separately because I really don’t want to misrepresent what the backend can do.
It also supports native training and inference paths for qualified architectures, full-parameter AdamW training, LoRA/PEFT fine-tuning, SafeTensors , checkpointing, and Hugging Face Hub integration for models, datasets, and tokenizers.
One of the parts I’m particularly proud of is the numerical validation work. I’ve been building reference comparisons to make sure the native Vulkan implementation behaves consistently with existing implementations rather than just producing outputs that look vaguely correct.
The current training parity results include 14 modern transformer architecture families passing strict tiny-model tests across two AdamW optimization steps, with maximum absolute parameter divergence below 2e-7.
Those are small deterministic validation fixtures, not full production-model benchmarks, but getting everything to agree that closely has been pretty satisfying. :3
Another thing I’m really passionate about is making ML workloads more practical on low-end and edge hardware.
I’ve been experimenting with GPUs like the RDNA3 graphics in my ROG Ally, along with much older hardware like Intel HD Graphics 520. There’s something incredibly satisfying about getting hardware that people tend to dismiss to actually do useful work.
A lot of my development time has gone into memory management, GPU synchronization, compute shader implementations, and figuring out ways to minimize overhead without sacrificing correctness.
I’m definitely not claiming Hierarchos-Native is faster than PyTorch or llama.cpp across the board. There’s still a ton of optimization and real-world compatibility work ahead of me, and I’m trying to approach that part carefully.
But I’m pretty happy with how far it’s gotten, especially considering how much of this has involved learning things the hard way and staring at Vulkan validation errors at completely unreasonable hours. x3
I wanted to share it here because Hugging Face interoperability is a major part of the project, and I’d really love to hear from people who have experience working on transformer implementations, GPU runtimes, or alternative training frameworks.
I’m particularly interested in what other developers think is missing from the current ecosystem when it comes to cross-vendor GPU acceleration and running ML workloads on resource-constrained hardware.
If anyone wants to poke through the implementation, try building it, suggest improvements, or just talk about Vulkan compute and ML infrastructure, I’d be super happy to chat! Here’s the repository:
Thanks for checking it out! I hope some of you find it interesting. :3