Xolotl (github.com/Indras-Mirror/Xolotl, MIT) is a descendant of Strata. The parent project's engine, server and core
kernels are still inside it - merged at 0.1.44.1 - but the author is candid that more than 70,000 lines have gone on
top: its own DeepSeek-V4.1 / V4-Flash and GLM-5.3-Flash engines, a Qwen3.8-27B dense engine grown from Mirko Perrone's port, an LFM3-Omni multimodal engine, a "fast mode", and the quantisation, expert-tiering and serving work in between.
It serves one OpenAI- and Anthropic-compatible endpoint across all of them.
The name is the project's own: Xolotl is Quetzalcoatl's dark twin in Aztec myth, the one who carries the dead up from
the underworld. The engine carries experts up from RAM and NVMe. That is the whole design in one sentence, and it is also
why this piece goes on the list of resource-constrained inference engines rather than the "big GPU" list.
Most engines tell you what model they support. Xolotl's README has a separate document for the machine budget, and the
numbers are unusual for being honest about which part of the system does the work.
The measured setup is one desktop: RTX 4090 (24 GB), Ryzen 7 5700X, 110 GB DDR4, one NVMe drive, Linux. The document
spells out what each of those budget items means:
That is a very different thing to "needs 24 GB of VRAM". The card is the least important number on that list, and the
authorship says so: community runs on 12 GB cards (an RTX 3060 with 46 GB of RAM reaches 149 tok/s prefill at 8K and
8-10 tok/s decode; an RTX 5070 gets 4.7 tok/s), while the same class of card runs Qwen3.8-Flash-Next at 39-48 tok/s. Same model family, same engine, four times the speed with a different quant - which is a reminder that in this work the
quant you choose is the memory plan.
A mixture-of-experts model only touches a few experts per token, so Xolotl keeps every expert somewhere: the hottest in
VRAM, tens of gigabytes in a pinned RAM arena that the GPU reads by DMA, the rest on NVMe read at the drive's ceiling -
and moves whichever experts the current token's router asks for, a layer ahead of time. That "a layer ahead" is the part
that separates it from a general runtime that pages weights in lazily and hopes the OS co-operates: an engine that owns
its own memory policy is the lesson my [Avalon](https://blog.localai.hex7.com/avalon-what-it-is-and-the-three-ways-it-differs-from-llama-cpp-and-strata/)
and [PolyStrata](https://blog.localai.hex7.com/polystrata-the-strata-fork-that-stopped-being-one-engine-and-what-that-means-against-llama-cpp-strata-and-avalon/)
notes made, and here it is again.
Per-model work on top of that: calibrated mixed-precision quantisation (the DeepSeek V4.1 build uses the GSQ family, which I
covered in GSQ and RCO), speculative decoding with exact sampling, a bit-exact 4-bit KV cache for 512K context, an arena that adapts to the
conversation, MTP speculative decoding for the Qwen models, GLM's abliteration LoRA on every path plus soft pruning.
Two features are worth reading for the engineering rather than the marketing:
That last paragraph is the one I'd point at if someone asked me what to learn from this project. A speed claim without an
agreement metric beside it is a claim I cannot use; here the trade is quantified in the same table as the speed.
Every number in that README comes from one machine, through the real server, while the model works its benchmark:
six hard agentic coding tasks graded by hidden tests, ten turns each, thinking enabled with a 32K budget. Decode speed is
measured during those agent tasks rather than in an idle generation loop - which is the only honest way to quote a
number for anything used by an agent harness.
It also scores its own models against the same models' hosted APIs on those tasks, and reports both directions: a local
DeepSeek-V4.1 build at 4.99/6 against that model's API score of 4.49; Qwen3.8-27B at 5.50 against an API 4.47. Where it
has not measured, it says "to come" - GLM-5.3-Flash is quoted as decode speed without a benchmark score, because there is
no run yet. And it retired its own HumanEval table for a reason: every model scored 19-21/21 on it, so the table no
longer separated them.
It is worth being clear about what this is not: research code from one developer, measured on one PC, with the invitation
"reports from other hardware welcome" in its own README. The Windows path exists only for DeepSeek-V4.1 and is native
MSVC + CUDA; everything quoted is Linux.
Xolotl: [https://github.com/Indras-Mirror/Xolotl](https://github.com/Indras-Mirror/Xolotl)
Machine requirements (the honest one): [https://github.com/Indras-Mirror/Xolotl/blob/main/docs/WILL_IT_RUN.md](https://github.com/Indras-Mirror/Xolotl/blob/main/docs/WILL_IT_RUN.md)
Benchmarks and how to measure your own PC: https://github.com/Indras-Mirror/Xolotl/blob/main/docs/xolotl/BENCHMARKS.md
Fast mode, what it costs: [https://github.com/Indras-Mirror/Xolotl/blob/main/docs/XOLOTL_FAST.md](https://github.com/Indras-Mirror/Xolotl/blob/main/docs/XOLOTL_FAST.md)
Changelog (with PR numbers): [https://github.com/Indras-Mirror/Xolotl/blob/main/docs/xolotl/CHANGELOG.md](https://github.com/Indras-Mirror/Xolotl/blob/main/docs/xolotl/CHANGELOG.md)
Write-ups by the author: [https://indrasmirror.au](https://indrasmirror.au)
llama.cpp: [https://github.com/ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp) · Strata: [https://github.com/Niko1221/Strata](https://github.com/Niko1221/Strata)