I've written before about getting started using Lemonade for local AI with VSCode and OpenClaw. Today I want to run through the CLI commands I use to get the most out of my Lemonade install, from installing custom models to benchmarking, using custom llama.cpp versions and more.
list #
Let's start at the beginning with a list of all the Lemonade approved models that you can install. You can browse what's available on the website but we're on the CLI today so we see them all with lemonade list. And of course list supports filtering with wildcards so we can see, for example, only the Gemma 4 MTP models:
lemonade list gemma*4*mtp
pull #
any of the approved models is easy: we can just lemonade load Gemma-4-12B-it-MTP-GGUF and the model is pulled and loaded. What if we want to load a Hugging Face model that hasn't been approved like this QAT version of Gemma? In that case we can simply point the lemonade pull command at a Hugging Face path:
lemonade pull unsloth/gemma-4-12B-it-qat-GGUF:UD-Q4_K_XL
load #
And once it's downloaded we can load the model by name with lemonade load.
lemonade load gemma-4-12B-it-qat-GGUF-UD-Q4_K_XL
status #
Now it's loaded, how about we check the status of our Lemonade server and the models it's loaded using lemonade status.
--ctx-size #
By default Lemonade will auto configure the context size (Ctx) for us and we can see it's set to 262144 for me in the screenshot above. But we can use the --ctx-size switch to set that manually.
lemonade load gemma-4-12B-it-qat-GGUF-UD-Q4_K_XL --ctx-size 131072
--llamacpp-args #
Most model providers on Hugging Face provide recommended best practices for running the model you've downloaded. For the Gemma model we're working with you can see them here and we can see that they recommend setting temperature, top_p and top_k for best results, we can pass these when we load the model with --llamacpp-args.
lemonade load gemma-4-12B-it-qat-GGUF-UD-Q4_K_XL --llamacpp-args "--temperature 1.0 --top_p 0.95 --top_k 64"
--save-options #
But we don't want to have to type out a complicated set of options every time we load this model so let's use the --save-options switch to set the default set of options for this model:
lemonade load gemma-4-12B-it-qat-GGUF-UD-Q4_K_XL --llamacpp-args "--temperature 1.0 --top_p 0.95 --top_k 64" --ctx-size 131072 --save-options
Now we can just do:
lemonade load gemma-4-12B-it-qat-GGUF-UD-Q4_K_XL
And our favourite options are automatically populated.
bench #
Now wouldn't it be cool if we could actually measure the performance of the model on our hardware, benchmark the time to first token, tokens per second and VRAM usage across different scenarios? You've guessed it, bench does exactly that:
lemonade bench gemma-4-12B-it-qat-GGUF-UD-Q4_K_XL --ctx-size 131072
Modalities, engines and backends, oh my! #
We're about to start working with these so let's define them quickly. Lemonade supports many modalities (kinds of input and output data), you can see them all in the Supported Configurations section of its readme. We've been working with the Text generation modality so far, and the supported engine, backends, devices and operating system configurations for that look like this today:
We can see that for Text generation Lemonade currently supports 6 Engines (llamacpp, llamacpp-hrx, flm, ryzenai-llm, vllm and ds4) and each Engine runs on a Backend, support for which is determined by a combination of our Device and OS. Each time you run a model, behind the scenes Lemonade will evaluate your hardware and OS and automatically select the best Engine and Backend combination. That's great for getting started but what if we want to control which Engine or Backend we're using?
Recipes for what are now called Engines. The rename hasn't been completed everywhere yet so I'm going to refer to Engines/Recipes from now on: they're the same thing.
backends
OK so let's take a look at what Engines/Recipes our current platform supports.
lemonade backends
As I'm running Ubuntu on a Strix Halo you can see I've got a lot of options available to install. Lemonade selected vulkan as the backend for llamacpp but I can see that rocm is also available so let's install that.
lemonade backends install llamacpp:rocm
--llamacpp #
With our new backend installed let's use it to run our model with the --llamacpp option.
lemonade load gemma-4-12B-it-qat-GGUF-UD-Q4_K_XL --llamacpp rocm --llamacpp-args "--temperature 1.0 --top_p 0.95 --top_k 64" --ctx-size 131072
And that bench command is now going to come in particularly handy because when we run it Lemonade will benchmark all the installed backends so we can compare performance across them.
lemonade bench gemma-4-12B-it-qat-GGUF-UD-Q4_K_XL --ctx-size 131072
And we can see that for this model Rocm is faster in TTFT and uses slightly less VRAM but Vulkan wins on TPS.
config #
We can get all Lemonade's config settings with, unsurprisingly, lemonade config. This will spit out every config setting. Below I'm showing a snip of the [llamacpp] settings it outputs for me but you'll see a lot more.
config set #
We can also set individual values in the config. I find this particularly useful if I want to test out a newer version of one of the backends, for example to switch to the latest upstream version of ROCm:
lemonade config set llamacpp.rocm_bin="latest"
This will update the config and, if necessary, download the binary so that next time I run a model with rocm it's going to use the latest version. And of course this works for other binaries and I can specify upstream tags, for example:
lemonade config set llamacpp.cuda_bin="b8664"
And if I want to go back to the built-in default I can just:
lemonade config set llamacpp.cuda_bin="builtin"
--help-all #
Finally, I wouldn't know any of this if it wasn't for the awesome lemonade CLI docs and the great help built into the CLI, which is available for every command.
lemonade --help-all
If I've missed your favourite CLI command or option let me know.