Source:
The RegisterHey Claude, optimize this model for me Even as AMD’s GPUs have grown more competitive, the House of Zen has struggled to shake the perception that its chips are less capable because they don’t run CUDA. At its Advancing AI event in San Francisco this week, AMD unveiled ROCm.AI, which promises to let users vibe code their way to faster
inferenceperformance. In reality, the so-called CUDA moat has become considerably shallower over the past few years as frameworks likePyTorchand JAX have made it possible for developers to write once and, for the most part, run anywhere without ever having to touch CUDA or AMD’s ROCm and HIP libraries. But just because the code runs, it doesn’t necessarily mean it’s performant. Low-level programming interfaces like CUDA and ROCm remain key to unlocking a chip’s true potential. However, hand tuning GPU kernels and general matrix-matrix multiplication (GEMM) routines to take full advantage of the silicon isn’t exactly something everyone has the experience necessary to do. But as it turns out, many of the same models developers are trying to optimize for are surprisingly good at it. “For every generation of AMD GPUs, we have published not just the ISA spec. We actually publish the machine-readable ISA,” said AMD corporate VP of AI software and solutions Anush Elangovan, adding that as a result, “the frontier models are very, very capable of programming to AMD’s hardware.” With ROCm.AI, AMD hopes to streamline this capability. The platform plugs into existing code assistants running on frontier models and provides them with the tools and documentation necessary to deploy, debug, and optimize models and serving frameworks for AMD Instinct hardware. One of these tools is an automated workload performanceoptimizationsystem called Hyperloom. When the tool is called, for example bypromptingthe code assistant to “optimize MiniMax M3 with Hyperloom,” it might spin up an inference server in a Docker container, run benchmarks to establish baseline performance, profile the workload to identify bottlenecks, and adjust the configuration or even generate custom CPU kernels on the fly, Elangovan explained. In testing on AMD’s newly launched Helios racks, this process, Elangovan claims, was able to boost model performance by 38 percent over baseline. “We want to give you the ability to eke out the maximum performance,” he said. “This makes it incredibly easy for anyone to consume, debug, profile, and deploy.” To further improve this process, AMD says that it’s leaning on its close relationship with AI model houses like OpenAI andAnthropicto ensure their models are trained to better understand the inner workings of both their hardware and software. “We’re not just using the frontier model to generate a kernel,” Elangovan said. “We’re working deeply with frontier model companies so that they natively speak AMD programming.” In addition to its built-in command-line interface, ROCm.AI will be offered as a plug-in for popular coding assistants, including Anthropic’sClaudeCode, OpenAI’s Codex, Google's Antigravity, andCursor. ®Get AI news in your inbox
Daily digest of what matters in AI.
Key Terms Explained #
Anthropic
An AI safety company founded in 2021 by former OpenAI researchers, including Dario and Daniela Amodei.
Claude
Anthropic's family of AI assistants, including Claude Haiku, Sonnet, and Opus.
CUDA
NVIDIA's parallel computing platform that lets developers use GPUs for general-purpose computing.
GPU
Graphics Processing Unit.