Amid the growing concern about the rising cost of tokens, AMD has released an open source inference server called Lemonade that shows how easily users can execute AI inferencing on their own computers.
“We’re building Lemonade because we really want a completely free, completely open source option for people to use on their AMD PCs if they want to do AI inference, and particularly if they want to connect that to a lot of other apps or even build it into their own apps,” said Jeremy Fowers, AMD machine learning engineer and one of the lead maintainers of Lemonade, explaining the software in an introductory video.
For Windows, the Lemonade installation simply involves down and installing a small MSI file, and then perusing a list of suggested models to download and then test, directly in the console. Not coincidentally, Lemonade showcases the AI capabilities of the latest AMD hardware, including the latest Ryzen CPUs and Radeon GPUs, as well as the company’s on-board Neural Processing Units (NPUs). You don’t need AMD processors to run Lemonade, but in the words of Matthew McConaughey, it would “be a lot cooler if you did.”
The software is “completely private,” Fowers said. “It never phones home, never does any telemetry.”
A Control Center for Local AI
Lemonade, a locally run generative AI server, runs either as a desktop app or a Web app. It provides a graphical grid interface where users can test different models for tasks such as text, image and speech generation.
With Lemonade, users can take new models such as Qwen or Z Image Turbo for test drives.
“This is how you kick the tires and get a feel for what each of the models can do,” Fowers said. “From there, you can start connecting to other apps or building your own app.”
For a developer, Lemonade can serve as a central hub for managing multiple models. AMD has curated about 150 models of interest, and users can load others as well. All the usual suspects in their latest incarnations are available, including Llama, Stable Diffusion and Qwen. AMD has created a few models of its own too, including the Ryzen AI software stack. Users can also import their own models or, via API, connect to the hosted models from OpenAI’s and Anthropic’s.
In addition to a testing palette, Lemonade also offers a section for back-end configuration. Lemonade takes care of much of the configuration work. The installation process probes the hardware and auto-configures the server for the specific environment. Users don’t have to worry about things like deciding whether to target a Radeon GPU or a Ryzen AI CPU–that’s all sorted out at installation time.
In addition to models, users can also import other apps that communicate by the OpenAI Rest or the Ollama API. Using the Open WebUI, for instance, you can build a chat application designed specifically for your home. Or you can build data-driven workflows with n8n. Build agents with Gaia, and games with Infinity Arcade.
Optimized for AMD Gear
Fowers readily admits that AMD is not trying to compete with other inference servers, such as Ollama and LM Studio, but rather to introduce the world to the capabilities of the latest AMD hardware. In particular, the project hopes that other local model providers can inspect the code and bake the AMD optimizations into their own software.
The software targets the latest AI-optimized AMD processors. These include the Ryzen AI 300, 400, and Max Series, select Ryzen 8000 and 7000 mobile series (though it excludes the regular Ryzen 7000 or 9000 series). The company’s NPUs in these models were designed specifically for large-scale low-power inferencing work.
Fowers explained that certain models, such as Qwen, run a lot faster and require less power on an NPU. “You can run Qwen on your computer without spinning up your fans or draining your battery,” Fowers said, adding that running Qwen on the NPU is a good way to continuously operate agents.
Lemonade also takes advantage of AMD’s ROCm.AI (Radeon Open Compute platform) GPU management stack (AMD’s CUDA equivalent, roughly). Lemonade will also work on Intel processors, though users won’t get the NPU/ROCm performance boost.
The development effort welcomes outside help. AMD itself is concentrating on writing the AMD-specific bits, but is open to contributions that target other platforms.
For Linux, the project maintains a Deb installer for Ubuntu/Debian. The community maintains installers for Arch and Fedora, as well as for the Snap package manager. A Mac installer is also maintained by the community. Docker-based installations are also available. Intel doesn’t have an equivalent to Lemonade, though users can enjoy many of the same capabilities with the OpenVINO Model Server or the UI-based Intel AI Playground.