Strix Halo tuned rust engine

An open-source MIT-licensed project called Talus released a Rust-based inference engine for AMD's Strix Halo (Ryzen AI Max+ 395 with 128 GB), forked from Niko1221's C++/HIP engine Strata, with GPU kernels re-tuned for the gfx1151 target. Talus serves one model at a time through an OpenAI-compatible API, with a terminal UI for switching models and presets, and was benchmarked end-to-end through the server on an HP ZBook Ultra G1a capped at 70 W running headless Linux 7.0 across Qwen3.8-27B, Qwen3.8-Flash-Next, GLM-5.3-Flash, MiMo-V2.6-Flash and Laguna-S-2.1. The code keeps per-GPU tables and multi-device hooks for possible RDNA4 and multi-box support, and a built-in benchmark and validation tool including SWE-bench with anti-cheating is planned for the next release.

Talus is an open-source MIT inference engine for Strix Halo Max+ 395/128G . The engine is a fork of Strata, a C++/HIP engine by Niko1221. We ported Strata’s host and server to Rust, tried a few other popular models, and re-tuned the GPU kernels, which come from Strata and, through it, from ggml/llama.cpp, for gfx1151. It serves one model at a time through an OpenAI-compatible API. A terminal UI switches models and presets, and explains trade offs when changing settings. Why? I started chasing the big MoE models that I could fit on this laptop, which spiraled into forking everycode base I could find for performance. The Goods: HP ZBook Ultra G1a, Ryzen AI Max+ 395, 128 GB, firmware-capped at 70 W, headless Linux 7.0 Qwen3.8-27B, Qwen3.8-Flash-Next, GLM-5.3-Flash, MiMo-V2.6-Flash, Laguna-S-2.1. Numbers: tok/s, end to end through the server, on an HP ZBook Ultra G1a capped at 70 W How we got here: What mattered for gfx1151: Honest Comparisons same laptop, measured by us Credits: Strata - Niko1221 : the base. ggml / llama.cpp: kernels and formats. The model teams: Qwen, Z.ai, Xiaomi MiMo, poolside. Unsloth: the GGUF quants z-lab: the DFlash drafters. Gufo: the long-copy drafting idea. AMD ROCm. What’s next: Talus is tuned and tested only on Strix Halo today. The code keeps per-GPU tables amd multi-device hooks, so other AMD GPUs RDNA4 and multi-box setups may come later. A built-in benchmark and validation tool quality suites plus SWE-bench with anti-cheating is coming in the next release. Github: nsbradley88/talus-public HuggingFace: j2ipper