IBM has made Spyre a first-class PyTorch backend, giving developers the same workflows across devices.
In October 2025, IBM and the PyTorch Foundation outlined a plan for deeper Spyre enablement in 2026: treating the accelerator as a first-class PyTorch backend and supporting multi-accelerator scheduling. The latest software ecosystem updates deliver on that promise. Together, torch-spyre, Spyre's vLLM plugin, compiler improvements, and continuous integration work are making it easier to bring familiar models and workflows to a different class of AI hardware.
AI is increasingly being embedded in the systems that process transactions, detect fraud, analyze claims, and assist employees. And with this growing usage, the AI hardware ecosystem is expanding. GPUs remain central to modern AI, but they are increasingly joined by accelerators with different architectures, execution models, and performance characteristics. Giving developers more choices requires more than building new chips. The software ecosystem must make new accelerators usable without forcing developers to abandon familiar models, tools, and workflows.
Spyre is IBM’s dataflow AI accelerator, optimized for inference. It is designed for enterprise teams running AI alongside their applications and data on IBM Z, LinuxONE, and Power systems. Its reduced-precision compute is well suited to the matrix-heavy work in language generation and embedding models. Unlike a conventional GPU, which executes kernels as instruction streams across scheduled groups of threads, Spyre triggers computation as data arrives at its compute engines. PyTorch provides the common layer that allows this different architecture to participate in an ecosystem built largely around GPUs.
Extending PyTorch beyond GPUs #
At the center of this effort is torch-spyre, a PyTorch backend that compiles ordinary PyTorch code into an executable plan for Spyre. Most of a model’s logic passes through the PyTorch compiler unchanged. When a model contains an operation that the underlying stack cannot yet lower effectively, an adapter can replace it with a mathematically equivalent form that the compiler can handle. The model’s computation remains the same, while its implementation becomes compatible with the accelerator.
The move to PyTorch’s Inductor backend also reduced the amount of custom compiler code that the team must maintain. The resulting compiler is roughly seven times smaller than the system it replaced and measurably less complex per function. That smaller surface area makes it easier to carry improvements from upstream PyTorch into torch-spyre and focus Spyre-specific work where it matters.
The same principle applies at the device boundary. In a recent demonstration, moving graph construction off the data-transfer path improved Granite 3.3 8B performance at batch size 1 and sequence length 1024 by 9.9% for prefill and 13.6% for decode. Removing that work from compute dispatch as well brought the total improvement to 1.7 times faster prefill and 2.4 times faster decode, with roughly three quarters of the gain coming from the compute path.
Preserving vLLM portability #
Spyre is an out-of-tree accelerator, rather than one of the GPU platforms most commonly supported by AI software. Spyre relies on TorchDynamo to trace a model graph and TorchInductor to lower that graph into representations optimized for the target hardware. Within vLLM, extension mechanisms such as CustomOp and PluggableLayer allow a hardware plugin to customize behavior without reimplementing large portions of the software stack.
Preserving these extension points matters as vLLM evolves. Some new model implementations are becoming increasingly specialized for particular models and frontier GPU hardware. To preserve portability alongside that optimization work, the vLLM community is developing hardware-agnostic layers that are isolated from hardware-specific paths and designed to be compilable, extensible, and portable across accelerator platforms. Early support has landed in vLLM, and the pathway has been validated with the Spyre plugin on models including Gemma 4, Qwen3, and Granite 4.2. On NVIDIA H100 GPUs, these hardware-agnostic layers delivered total token throughput within 3.4% of the native implementation, measured as the geometric mean across three recent models.
One workflow, more hardware choices #
For IBM Z, LinuxONE, and Power11 users, the intended experience is straightforward: Obtain models from ecosystems like Hugging Face and use substantially the same PyTorch-based workflows they would use with Nvidia GPUs. Shared PyTorch extension points hide much of the architectural difference beneath the framework, reducing adoption friction and avoiding an experience limited to a small catalog of specially packaged model weights. Consider an insurance-claims workflow that combines XGBoost models for fraud detection with a Gemma 26B model on Spyre for forensic analysis. The software stack enables developers to assemble the workflow using familiar model ecosystems and framework interfaces, while the hardware is chosen for the work it suits.
Spyre's PyTorch integration is also how it connects to llm-d. llm-d runs distributed inference on top of vLLM and is built to support multiple accelerator types, not a single hardware path. vLLM reaches each accelerator through PyTorch: Once a device has a working PyTorch backend, a vLLM hardware plugin can expose it, and then llm-d can schedule work onto it. Spyre follows this path. torch-spyre provides the PyTorch backend, and the Spyre vLLM plugin builds on top of it. As a result, the work that makes Spyre a PyTorch device is the same work that makes it usable in llm-d deployments.
Making compatibility measurable #
Broad compatibility is only useful when users have clear evidence that models run reliably across platforms. IBM Research engineers are building that evidence through end-to-end model testing and continuous integration. This work helps the team detect when upstream framework changes affect downstream hardware support.
End-to-end model testing is critical because failures can arise from combinations of operations, shapes, layouts, numerical ranges, and compiler fusion behavior that isolated unit tests do not reveal. Running real models against trusted CPU or GPU references turns each model into both a compatibility test and a probe for hardening the platform. It also gives developers clearer signals about which models and accelerator configurations have been exercised together.
In the second half of 2026, IBM, Red Hat, and their collaborators in the PyTorch Foundation including Meta and Hugging Face have pushed new software ecosystem updates covering day-one model enablement, hardware-agnostic models in vLLM, continuous regression and release testing, and native-device integration. A new demonstration on the frontend and backend compiler explains more of the stack. Together, this body of work shows how model adapters, compiler paths, runtime integration, vLLM extensions, and CI/CD combine to support Spyre as a first-class participant in the PyTorch ecosystem.
By combining familiar workflows, portable compiler paths, broad model enablement, and transparent validation, IBM is helping make hardware choice a practical option rather than a software integration burden.
Related posts #
- NewsKim Martineau
- llm-d 1.0 hardens open-source AI inference for day 2 production operations ReleasePeter Hess
- A new IBM Research specialization teaches how AI models run on modern accelerators Technical noteKaoutar El Maghraoui
- Preparing chip designers for the nanostack era ReleasePeter Hess