Fresher Lemonade: CLI tips and tricks

A CLI walkthrough of Lemonade, the local AI server, documents the commands for installing, loading, benchmarking and configuring models, including `lemonade list`, `lemonade pull`, `lemonade load`, `lemonade status` and `lemonade bench`. The guide shows pulling an unapproved Hugging Face model with `lemonade pull unsloth/gemma-4-12B-it-qat-GGUF:UD-Q4_K_XL`, overriding the auto-configured context size of 262144 via `--ctx-size 131072`, passing `--llamacpp-args "--temperature 1.0 --top_p 0.95 --top_k 64"`, and persisting those defaults with `--save-options`. Lemonade's text-generation modality currently supports 6 engines — llamacpp, llamacpp-hrx, flm, ryzenai-llm, vllm and ds4 — each running on a backend.

I've written before about getting started using Lemonade https://lemonade-server.ai/?ref=admcpr.com for local AI with VSCode https://admcpr.com/easy-local-copilot-with-vs-code-and-lemonade/ and OpenClaw https://admcpr.com/local-agents-with-lemonade-and-openclaw/ . Today I want to run through the CLI commands I use to get the most out of my Lemonade install, from installing custom models to benchmarking, using custom llama.cpp versions and more. list Let's start at the beginning with a list of all the Lemonade approved models https://lemonade-server.ai/models.html?ref=admcpr.com that you can install. You can browse what's available on the website but we're on the CLI today so we see them all with lemonade list . And of course list supports filtering with wildcards so we can see, for example, only the Gemma 4 MTP models: lemonade list gemma 4 mtp pull Loading any of the approved models is easy: we can just lemonade load Gemma-4-12B-it-MTP-GGUF and the model is pulled and loaded. What if we want to load a Hugging Face model that hasn't been approved like this QAT version of Gemma https://huggingface.co/unsloth/gemma-4-12B-it-qat-GGUF?ref=admcpr.com ? In that case we can simply point the lemonade pull command at a Hugging Face path: lemonade pull unsloth/gemma-4-12B-it-qat-GGUF:UD-Q4 K XL load And once it's downloaded we can load the model by name with lemonade load . lemonade load gemma-4-12B-it-qat-GGUF-UD-Q4 K XL status Now it's loaded, how about we check the status of our Lemonade server and the models it's loaded using lemonade status . --ctx-size By default Lemonade will auto configure the context size Ctx for us and we can see it's set to 262144 for me in the screenshot above. But we can use the --ctx-size switch to set that manually. lemonade load gemma-4-12B-it-qat-GGUF-UD-Q4 K XL --ctx-size 131072 --llamacpp-args Most model providers on Hugging Face provide recommended best practices for running the model you've downloaded. For the Gemma model we're working with you can see them here https://huggingface.co/unsloth/gemma-4-12B-it-qat-GGUF?ref=admcpr.com best-practices and we can see that they recommend setting temperature , top p and top k for best results, we can pass these when we load the model with --llamacpp-args . lemonade load gemma-4-12B-it-qat-GGUF-UD-Q4 K XL --llamacpp-args "--temperature 1.0 --top p 0.95 --top k 64" --save-options But we don't want to have to type out a complicated set of options every time we load this model so let's use the --save-options switch to set the default set of options for this model: lemonade load gemma-4-12B-it-qat-GGUF-UD-Q4 K XL --llamacpp-args "--temperature 1.0 --top p 0.95 --top k 64" --ctx-size 131072 --save-options Now we can just do: lemonade load gemma-4-12B-it-qat-GGUF-UD-Q4 K XL And our favourite options are automatically populated. bench Now wouldn't it be cool if we could actually measure the performance of the model on our hardware, benchmark the time to first token, tokens per second and VRAM usage across different scenarios? You've guessed it, bench does exactly that: lemonade bench gemma-4-12B-it-qat-GGUF-UD-Q4 K XL --ctx-size 131072 Modalities, engines and backends, oh my We're about to start working with these so let's define them quickly. Lemonade supports many modalities kinds of input and output data , you can see them all in the Supported Configurations https://github.com/lemonade-sdk/lemonade?ref=admcpr.com supported-configurations section of its readme. We've been working with the Text generation modality so far, and the supported engine, backends, devices and operating system configurations for that look like this today: We can see that for Text generation Lemonade currently supports 6 Engines llamacpp, llamacpp-hrx, flm, ryzenai-llm, vllm and ds4 and each Engine runs on a Backend, support for which is determined by a combination of our Device and OS. Each time you run a model, behind the scenes Lemonade will evaluate your hardware and OS and automatically select the best Engine and Backend combination. That's great for getting started but what if we want to control which Engine or Backend we're using? Recipes for what are now called Engines . The rename hasn't been completed everywhere yet so I'm going to refer to Engines/Recipes from now on: they're the same thing. backends OK so let's take a look at what Engines/Recipes our current platform supports. lemonade backends As I'm running Ubuntu on a Strix Halo you can see I've got a lot of options available to install. Lemonade selected vulkan as the backend for llamacpp but I can see that rocm is also available so let's install that. lemonade backends install llamacpp:rocm --llamacpp With our new backend installed let's use it to run our model with the --llamacpp option. lemonade load gemma-4-12B-it-qat-GGUF-UD-Q4 K XL --llamacpp rocm --llamacpp-args "--temperature 1.0 --top p 0.95 --top k 64" --ctx-size 131072 And that bench command is now going to come in particularly handy because when we run it Lemonade will benchmark all the installed backends so we can compare performance across them. lemonade bench gemma-4-12B-it-qat-GGUF-UD-Q4 K XL --ctx-size 131072 And we can see that for this model Rocm is faster in TTFT and uses slightly less VRAM but Vulkan wins on TPS. config We can get all Lemonade's config settings with, unsurprisingly, lemonade config . This will spit out every config setting. Below I'm showing a snip of the llamacpp settings it outputs for me but you'll see a lot more. config set We can also set individual values in the config. I find this particularly useful if I want to test out a newer version of one of the backends, for example to switch to the latest upstream version of ROCm: lemonade config set llamacpp.rocm bin="latest" This will update the config and, if necessary, download the binary so that next time I run a model with rocm it's going to use the latest version. And of course this works for other binaries and I can specify upstream tags, for example: lemonade config set llamacpp.cuda bin="b8664" And if I want to go back to the built-in default I can just: lemonade config set llamacpp.cuda bin="builtin" --help-all Finally, I wouldn't know any of this if it wasn't for the awesome lemonade CLI https://lemonade-server.ai/docs/guide/cli/?ref=admcpr.com docs and the great help built into the CLI, which is available for every command. lemonade --help-all If I've missed your favourite CLI command or option let me know. https://admcpr.com/contact/