{"slug": "apple-silicon-inference-engine-kapitsa", "title": "Apple Silicon Inference Engine: Kapitsa", "summary": "Kapitsa, an inference engine for Apple silicon from developer nace-ai, reports the fastest decode in 32 of 36 model-and-Mac pairs against nine competing engines, reaching up to 2.9× MLX, according to the project's published benchmarks. The engine runs Qwen, Nemotron and Recursive's Flint models locally and launches coding agents including opencode, Claude Code, Codex and pi on a local model while serving OpenAI and Anthropic APIs; on a 16 GB M1 Pro it decodes Flint at 61 tok/s versus llama.cpp's 43 on the same 2-bit GGUF. Kapitsa's KFlash tree speculative decoding reaches up to 3.5× llama.cpp and 2.2× oMLX, and its SSD prefix cache starts a 16K-token follow-up on the 35B model in 0.16 s instead of 17 s.", "body_md": "Kapitsa is an inference engine for Apple silicon that runs Qwen, Nemotron and Recursive's Flint models locally, built for coding agents. It starts opencode, Claude Code, Codex or pi on a local model, and serves OpenAI and Anthropic APIs.\n\n[Quickstart](#quickstart) · [CLI](#cli) · [Benchmarks](#benchmarks) · [Models](#models) · [How KFlash works](https://github.com/nace-ai/kapitsa/blob/main/docs/kflash.md) · [Docs](https://github.com/nace-ai/kapitsa/blob/main/docs/usage.md)\n\n- [2026/10] 🪶 **Flint on a 16 GB Mac: decodes at 61 tok/s vs llama.cpp's 43 on the same 2-bit GGUF.**`--model flint` takes unsloth's 2-bit GGUF on 16-24 GB Macs: its routed experts run as stored, the rest requantized to 3-6 bits. 61 tok/s decode on an M1 Pro, in 11.6 GB ([numbers](https://github.com/nace-ai/kapitsa/blob/main/docs/benchmarks.md#recursive-flint-40b-a25b-with-2-bit-experts-m1-pro-16-gb-2026-10-22) ).\n- [2026/10] 🪨 **Recursive's Flint 40B-A2.5B: Kapitsa reads prompts faster than Recursive's own engine and decodes faster than llama.cpp.** On the M4 Pro every Kapitsa run read 4K and 16K prompts faster than every run of rec 0.1.0. On the M5 Pro, since Flint's attention moved to the tensor units (#345), both runs read 4K and 16K prompts faster than rec's and llama.cpp's; earlier builds read slower, and both runs carry harness flags. On both Macs every Kapitsa run decodes faster than llama.cpp ([M4 Pro](https://github.com/nace-ai/kapitsa/blob/main/docs/benchmarks.md#recursive-flint-40b-a25b-m4-pro-2026-10-22) ,[M5 Pro](https://github.com/nace-ai/kapitsa/blob/main/docs/benchmarks.md#recursive-flint-40b-a25b-m5-pro-2026-10-22) ).\n- [2026/10] 🚀 **KFlash: tree speculative decoding, up to 3.5× llama.cpp and 2.2× oMLX.** One verify pass checks a whole tree of the drafter's guesses ([how it works](https://github.com/nace-ai/kapitsa/blob/main/docs/kflash.md) ).\n- [2026/10] 🏆 **Kapitsa: the fastest decode in 32 of 36 model and Mac pairs** against nine engines, up to 2.9× MLX ([poster](#benchmarks) ).\n- [2026/10] 🤝 **KFlash budget: subagents decode together.** Three requests at once share one verify pass, up to**+23% total throughput** on the 27B.\n- [2026/10] ⚡ **QMT prefill: read the prompt up to 1.5× faster on M5.** Metal 4 matrix units for the prompt (4B, M5 Pro).\n- [2026/10] 💾 **SSD prefix cache: restart the server, keep the conversation.** A 16K-token follow-up on the 35B starts in**0.16 s instead of 17 s** , about 100× (`--disk-cache` ).\n- [2026/10] 🧠 **Prefix reuse: subagents start warm.** A new question over the shared system prompt answers in**0.06 s instead of 1.9 s** (4K prompt, 4B, M5 Pro).\n- [2026/10] 🧪 **Output check: speed has to prove it kept the answers.** A default changes only after its logits stay within twice the model's noise floor ([how](https://github.com/nace-ai/kapitsa/blob/main/CONTRIBUTING.md#output-check) ).\n- [2026/10] 📊 **`kapitsa bench compare`: see it on your own Mac.** The same test against every engine installed here, in one command.\n\n**Requirements:** a Mac with Apple silicon (M1 or newer) and macOS 14 Sonoma or later.\n\n```\nbrew install nace-ai/tap/kapitsa\n# or: curl -fsSL https://raw.githubusercontent.com/nace-ai/kapitsa/main/install.sh | bash\nkapitsa opencode --model 35b    # also: claude, codex, pi\n```\n\n<sub>On a 16-24 GB Mac use `--model 9b`, or `4b` for the quickest replies.</sub>\n\n```\nkapitsa serve                   # chat + OpenAI/Anthropic APIs at http://127.0.0.1:8000\nkapitsa claude                  # Claude Code on Kapitsa\nkapitsa doctor                  # this Mac's GPU and memory, and which models fit it\nkapitsa cluster                 # the Macs on this network that run Kapitsa, and their models\nkapitsa bench                   # benchmark this Mac, share it with the community\nkapitsa bench compare           # the same test against the other engines installed here\nkapitsa pull 27b                # download a model without starting it\nkapitsa list                    # downloaded models and their size\nkapitsa rm 27b                  # delete a downloaded model\nkapitsa import model.gguf       # use a Qwen or Flint GGUF from LM Studio, Ollama or llama.cpp\nkapitsa update                  # update now (it also updates itself)\n```\n\n`--model` picks the model and its precision (`35b:6bit`). `kapitsa <command> --help` has the options.\nA Mac that cannot hold a model whole uses another Mac on the network that runs it, and two Macs that would each stream\nFlash-Next pair to hold it in memory between them ([Several Macs](https://github.com/nace-ai/kapitsa/blob/main/docs/usage.md#several-macs)); `--private` keeps a\nserver to its own Mac. [Usage guide](https://github.com/nace-ai/kapitsa/blob/main/docs/usage.md)\n\nKapitsa has the fastest decode in 32 of the 36 model and Mac pairs measured against nine engines ([how each number was measured](https://github.com/nace-ai/kapitsa/blob/main/docs/benchmarks.md#how-the-readme-poster-was-measured)).\n\n**Each cell reads prefill / decode in tokens per second.** Prefill is a 4K-token prompt, decode a 512-token answer at T 0.6 (Flint at its card's T 1.0). × cannot run that model · — not measured. How these were measured, run dates and flags: [docs/benchmarks.md](https://github.com/nace-ai/kapitsa/blob/main/docs/benchmarks.md#how-the-readme-poster-was-measured).\n\nThe poster's numbers, and the models it leaves out, are in each Mac's table under [Models](#models). Every recorded run, with its Mac, model, prefill and decode speed, is in the [benchmark table](https://github.com/nace-ai/kapitsa/blob/main/bench/README.md). Measure your Mac: `kapitsa bench --submit` adds it to the [community results](https://github.com/nace-ai/kapitsa/blob/main/bench/community/RESULTS.md). Another engine faster on your Mac? `kapitsa bench compare` ranks every engine you have installed, and `--report` writes a Markdown report.\n\nThe catalog models measured on each Mac, against the engines that run them: one collapsed section per Mac, with a table for each model family. Decode is a 512-token coding answer at T 0.6 (Flint at its card's T 1.0), prefill a 4K-token prompt. Under each model's name are the weights Kapitsa ran; the other engines' weights and how each number was measured are in [docs/benchmarks.md](https://github.com/nace-ai/kapitsa/blob/main/docs/benchmarks.md#the-readmes-tables). Flint's speed from 1K to 128K tokens of prompt on the M5 Pro, against llama.cpp: [bench/results/2026-10-24-flint-m5pro-context](https://github.com/nace-ai/kapitsa/blob/main/bench/results/2026-10-24-flint-m5pro-context/README.md).\n\n<sub>Each cell: prefill / decode, tokens/s (a lone number is decode: no prefill run) · 🟩 fastest decode · ≈ a decode tie with Kapitsa · small type: the decoding method (none: no speculative decoding) · × can't run on that Mac · — not measured</sub>\n\n## **MacBook Pro (16-inch, 2026) · M5 Max · 128 GB** · 7 models\n\n| Model | **Kapitsa** | <sub>mlx‑serve 26.10.2</sub> | <sub>oMLX 0.7.1rc1</sub> | <sub>Splash 1.4.0</sub> | <sub>Ollama 0.40.5 / 0.40.6</sub> | <sub>MTPLX 2.12.2</sub> | <sub>MLX mlx-lm 0.32.2, mlx-vlm 0.7.8</sub> | <sub>llama.cpp b11669</sub> | <sub>LM Studio MLX runtime 1.13.1, llama.cpp runtime 2.60.0</sub> | \n|---|---|---|---|---|---|---|---|---|---|\n| **Qwen3.8-27B** ·`27b`<sub>dense · 4-bit, 20.2 GB</sub> | 🟩 459 / **140**<sub>KFlash</sub> | 690 / 130 <sub>DFlash2</sub> | 761 / 93 <sub>DFlash2</sub> | 898 / 116 <sub>DFlash2</sub> | 739 / 68 | 733 / 69 <sub>MTP</sub> | 784 / 54 <sub>MTP</sub> | 516 / 51 <sub>MTP</sub> | 217 / 32 | \n| **Qwen3.6-35B-A3B** ·`35b`<sub>MoE, 3B active · 4-bit, 22.1 GB</sub> | 🟩 4702 / **325**<sub>KFlash</sub> | 3818 / 264 <sub>MTP</sub> | 2968 / 232 <sub>DFlash2</sub> | 6640 / 290 <sub>DFlash2</sub> | 3690 / 166 | 4328 / 123 <sub>MTP</sub> | 4271 / 132 <sub>DFlash2</sub> | 2881 / 171 <sub>MTP</sub> | 1873 / 127 | \n| **Qwen3.8-Flash-Next 180B-A7B** ·`flash-next`<sub>MoE, 7B active · 4-bit, 105.9 GB</sub> | 1165 / 106 <sub>KFlash</sub> | 🟩 1720 / **108** | 1260 / 107 <sub>MTP</sub> | × | 60 | 1263 / 88 <sub>MTP</sub> | 1030 / 77 <sub>MTP</sub> | 842 / 96 | × | \n| **Nemotron Dory1.0 4B** ·`dory`<sub>looped dense · NVFP4, 3.2 GB</sub> | 🟩 **119**<sub>KFlash</sub> | × | × | × | × | × | 76 | 9 | × | \n| **Nemotron 3.5 Lightning 30B-A3B** ·`lightning`<sub>hybrid MoE, 3B active · NVFP4, 22.9 GB</sub> | 🟩 **267**<sub>KFlash</sub> | 190 | 190 | × | 153 | 131 | 139 | 159 | 110 | \n| **Nemotron 3.5 Bolt 74B-A9B** ·`bolt`<sub>hybrid MoE, 9B active · 2/4/8-bit mix, 25.1 GB</sub> | 🟩 1016 / **153**<sub>KFlash</sub> | × | 913 / 69 | × | × | × | 941 / 69 | × | × | \n| **Nemotron 3.5 Super-VL 120B-A12B** ·`super`<sub>hybrid MoE, 12B active · NVFP4, 76.4 GB</sub> | 🟩 **94**<sub>KFlash</sub> | × | 52 | × | 64 | 44 | 46 | 66 | 61 | \n\n## **MacBook Pro (16-inch, 2026) · M5 Pro · 48 GB** · 6 models\n\n**Qwen and Nemotron**\n\n| Model | **Kapitsa** | <sub>mlx‑serve 26.10.2</sub> | <sub>oMLX 0.7.1rc1</sub> | <sub>Splash 1.4.0</sub> | <sub>Ollama 0.40.5 / 0.40.6</sub> | <sub>MTPLX 2.12.2</sub> | <sub>MLX mlx-lm 0.32.2, mlx-vlm 0.7.8</sub> | <sub>llama.cpp b11669</sub> | <sub>LM Studio MLX runtime 1.13.1, llama.cpp runtime 2.60.0</sub> | \n|---|---|---|---|---|---|---|---|---|---|\n| **Qwen3.8-27B** ·`27b`<sub>dense · 4-bit, 20.2 GB</sub> | 606 / 77 <sub>KFlash</sub> | 🟩 464 / **78**<sub>DFlash2</sub> | 446 / 51 <sub>DFlash2</sub> | 674 / 72 <sub>DFlash2</sub> | 470 / 40 <sub>MTP</sub> | 462 / 41 <sub>MTP</sub> | 421 / 32 <sub>MTP</sub> | 322 / 30 <sub>MTP</sub> | 118 / 17 | \n| **Qwen3.6-35B-A3B** ·`35b`<sub>MoE, 3B active · 4-bit, 22.1 GB</sub> | 🟩 2802 / **201**<sub>KFlash</sub> | 2494 / 156 <sub>MTP</sub> | 1619 / 144 <sub>DFlash2</sub> | 3514 / 181 <sub>DFlash2</sub> | 1518 / 102 <sub>MTP</sub> | 2569 / 89 <sub>MTP</sub> | 2474 / 98 <sub>DFlash2</sub> | 1755 / 107 <sub>MTP</sub> | 997 / 96 | \n| **Nemotron Dory1.0 4B** ·`dory`<sub>looped dense · NVFP4, 3.2 GB</sub> | 🟩 **70**<sub>KFlash</sub> | × | × | × | × | × | 53 | 13 | × | \n| **Nemotron 3.5 Lightning 30B-A3B** ·`lightning`<sub>hybrid MoE, 3B active · NVFP4, 22.9 GB</sub> | 🟩 3141 / **159**<sub>KFlash</sub> | 1728 / 115 <sub>MTP</sub> | 1938 / 124 <sub>MTP</sub> | × | 2369 / 92 | 1743 / 101 | 1881 / 94 | 1915 / 101 <sub>MTP</sub> | 1479 / 72 | \n| **Nemotron 3.5 Bolt 74B-A9B** ·`bolt`<sub>hybrid MoE, 9B active · 2/4/8-bit mix, 25.1 GB</sub> | 🟩 **79**<sub>KFlash</sub> | × | 39 | × | × | × | 41 | × | × | \n\n**Recursive Flint**\n\n| Model | **Kapitsa** | <sub>llama.cpp b11804</sub> | <sub>rec 0.1.0</sub> | \n|---|---|---|---|\n| **Recursive Flint 40B-A2.5B** ·`flint`<sub>MoE, 2.5B active · 4-bit, 22.6 GB</sub> | 🟩 2691 / **149**<sub>KFlash</sub> | 2072 / 92 | 2138 / 145 | \n\n<sub>Qwen3.8-Flash-Next 180B-A7B does not fit in 48 GB, so both engines stream its experts from the SSD. With Q2_0 weights on both and Kapitsa's expert store pinned at 24 GB (the auto size here is about 29.9 GB, but four runs at that size were stopped at memory pressure), Kapitsa decodes faster than llama.cpp up to 4K tokens of context and slower at 16K and 32K, and its prefill is behind on short prompts, level at 4K and 16K and 2.7% ahead at 32K: not the fastest everywhere ([numbers and caveats](https://github.com/nace-ai/kapitsa/blob/main/docs/benchmarks.md#qwen38-flash-next-streamed-from-the-ssd-kapitsa-against-llamacpp-m5-pro-48-gb-2026-10-24)).</sub>\n\n## **MacBook Pro (16-inch, 2024) · M4 Pro · 48 GB** · 7 models\n\n**Qwen and Nemotron**\n\n| Model | **Kapitsa** | <sub>mlx‑serve 26.10.2</sub> | <sub>oMLX 0.7.1rc1</sub> | <sub>Splash 1.4.0</sub> | <sub>Ollama 0.40.5 / 0.40.6</sub> | <sub>MTPLX 2.12.2</sub> | <sub>MLX mlx-lm 0.32.2, mlx-vlm 0.7.8</sub> | <sub>llama.cpp b11669</sub> | <sub>LM Studio MLX runtime 1.13.1, llama.cpp runtime 2.60.0</sub> | \n|---|---|---|---|---|---|---|---|---|---|\n| **Qwen3.5-4B** ·`4b`<sub>dense · 4-bit, 4.6 GB</sub> | 🟩 1930 / **201**<sub>KFlash</sub> | 874 / 137 <sub>MTP</sub> | 803 / 102 <sub>MTP</sub> | × | 758 / 109 <sub>MTP</sub> | 780 / 104 <sub>MTP</sub> | 843 / 88 <sub>MTP</sub> | 650 / 70 <sub>MTP</sub> | 801 / 81 | \n| **Qwen3.8-27B** ·`27b`<sub>dense · 4-bit, 20.2 GB</sub> | 🟩 396 / **63**<sub>KFlash</sub> | 129 / 54 <sub>DFlash2</sub> | 136 / 45 <sub>DFlash2</sub> | 53 <sub>DFlash2</sub> | 124 / 32 <sub>MTP</sub> | 126 / 36 <sub>MTP</sub> | 131 / 28 <sub>MTP</sub> | 84 / 18 <sub>MTP</sub> | 127 / 15 | \n| **Qwen3.6-35B-A3B** ·`35b`<sub>MoE, 3B active · 4-bit, 22.1 GB</sub> | 🟩 1168 / **161**<sub>KFlash</sub> | 986 / 124 <sub>MTP</sub> | 1035 / 127 <sub>DFlash2</sub> | 135 <sub>DFlash2</sub> | 892 / 89 <sub>MTP</sub> | 918 / 80 <sub>MTP</sub> | 985 / 86 <sub>DFlash2</sub> | 663 / 69 <sub>MTP</sub> | 957 / 86 | \n| **Nemotron Dory1.0 4B** ·`dory`<sub>looped dense · NVFP4, 3.2 GB</sub> | 🟩 **64**<sub>KFlash</sub> | × | × | × | × | × | 48 | 6 | × | \n| **Nemotron 3.5 Lightning 30B-A3B** ·`lightning`<sub>hybrid MoE, 3B active · NVFP4, 22.9 GB</sub> | 🟩 945 / **112**<sub>KFlash</sub> | — | — | × | — | — | 721 / 65 <sub>DSpark</sub> | — | — | \n| **Nemotron 3.5 Bolt 74B-A9B** ·`bolt`<sub>hybrid MoE, 9B active · 2/4/8-bit mix, 25.1 GB</sub> | 🟩 519 / **69**<sub>KFlash</sub> | — | 298 / 35 | × | — | — | 300 / 32 | × | — | \n\n**Recursive Flint**\n\n| Model | **Kapitsa** | <sub>llama.cpp b11804</sub> | <sub>rec 0.1.0</sub> | \n|---|---|---|---|\n| **Recursive Flint 40B-A2.5B** ·`flint`<sub>MoE, 2.5B active · 4-bit, 22.6 GB</sub> | 🟩 968 / **128**<sub>KFlash</sub> | 977 / 80 | 796 / 127 | \n\n## **MacBook Pro (16-inch, 2023) · M3 Pro · 36 GB** · 10 models\n\n**Qwen and Nemotron**\n\n| Model | **Kapitsa** | <sub>Ollama 0.40.8</sub> | <sub>MTPLX 2.12.2</sub> | <sub>MLX mlx-lm 0.32.2, mlx-vlm 0.7.8</sub> | <sub>llama.cpp b11669</sub> | \n|---|---|---|---|---|---|\n| **Qwen3.5-4B** ·`4b`<sub>dense · 4-bit, 4.6 GB</sub> | 🟩 703 / **120**<sub>KFlash</sub> | 600 / 67 <sub>MTP</sub> | 612 / 64 <sub>MTP</sub> | 642 / 56 <sub>MTP</sub> | 512 / 50 <sub>MTP · b11804</sub> | \n| **Qwen3.5-9B** ·`9b`<sub>dense · 4-bit, 9.0 GB</sub> | 🟩 381 / **77**<sub>KFlash</sub> | — | — | 353 / 27 | 307 / 23 <sub>b11804</sub> | \n| **Qwen3.8-27B** ·`27b`<sub>dense · 4-bit, 20.2 GB</sub> | 🟩 110 / **40**<sub>KFlash</sub> | 99 / 23 <sub>MTP</sub> | 99 / 22 <sub>MTP</sub> | 102 / 18 <sub>MTP</sub> | 86 / 16 <sub>MTP · b11804</sub> | \n| **Qwen3.6-35B-A3B** ·`35b`<sub>MoE, 3B active · 4-bit, 22.1 GB</sub> | 🟩 855 / **106**<sub>KFlash</sub> | 681 / 58 <sub>MTP</sub> | 671 / 53 <sub>MTP</sub> | 695 / 53 <sub>DFlash2</sub> | 578 / 53 <sub>MTP · b11804</sub> | \n| **Nemotron Dory1.0 4B** ·`dory`<sub>looped dense · NVFP4, 3.2 GB</sub> | 🟩 450 / **36**<sub>KFlash</sub> | × | × | 344 / 30 | — | \n| **Nemotron 3 Nano 4B** ·`nano-4b`<sub>hybrid Mamba-2 · FP8, 5.3 GB</sub> | 765 / 34 <sub>KFlash</sub> | — | — | — | 🟩 413 / **35**<sub>Q4_K_M, b11804</sub> | \n| **Nemotron Nano 9B v2** ·`nano-9b`<sub>hybrid Mamba-2 · NVFP4, 7.9 GB</sub> | 330 / 21 <sub>KFlash</sub> | — | — | 🟩 **25**<sub>4-bit</sub> | — | \n| **Nemotron 3.5 Lightning 30B-A3B** ·`lightning`<sub>hybrid MoE, 3B active · NVFP4, 22.9 GB</sub> | 🟩 788 / **75**<sub>KFlash</sub> | — | — | 514 / 39 <sub>DSpark</sub> | — | \n| **Nemotron 3.5 Bolt 74B-A9B** ·`bolt`<sub>hybrid MoE, 9B active · 2/4/8-bit mix, 25.1 GB</sub> | 🟩 256 / **50**<sub>KFlash</sub> | — | — | — | — | \n\n**Recursive Flint**\n\n| Model | **Kapitsa** | <sub>llama.cpp b11804</sub> | \n|---|---|---|\n| **Recursive Flint 40B-A2.5B** ·`flint`<sub>MoE, 2.5B active · 4-bit, 22.6 GB</sub> | 🟩 801 / **74**<sub>KFlash</sub> | 772 / 51 | \n\n<sub>macOS 14.1, on AC power: Kapitsa's macOS 14 kernels (macOS 15 and later run fused ones). mlx-serve, oMLX and Splash need a newer macOS (×); LM Studio was not installed; rec 0.1.0 refuses this Mac: it requires 48 GiB. Each cell but Flint's is one run of the M4 Pro's protocol (Flint's is the mean of two runs, #341), the other engines in the M4 Pro's setup (not re-tuned here). Nano 4B and 9B v2: Kapitsa runs NVIDIA's published checkpoints, which keep some layers in BF16/FP8, so it reads more per token than the 4-bit files: the 4B's FP8 is ~1 byte per weight against Q4_K_M's ~0.6, the 9B's NVFP4 ~6.11 GB per token against 4.67 GB for mlx-lm's 4-bit. Nano 9B v2 on mlx-lm: no prefill shown, because its template ignores enable_thinking and the timed first token is a thinking token.</sub>\n\n## **MacBook Pro (14-inch, 2021) · M1 Pro · 16 GB** · 7 models\n\n**Qwen and Nemotron**\n\n| Model | **Kapitsa** | <sub>mlx‑serve 26.10.2</sub> | <sub>oMLX 0.7.1rc1</sub> | <sub>Ollama 0.40.5 / 0.40.6</sub> | <sub>MTPLX 2.12.2</sub> | <sub>MLX mlx-lm 0.32.2, mlx-vlm 0.7.8</sub> | <sub>llama.cpp b11669</sub> | <sub>LM Studio MLX runtime 1.13.1, llama.cpp runtime 2.60.0</sub> | \n|---|---|---|---|---|---|---|---|---|\n| **Qwen3.5-4B** ·`4b`<sub>dense · 4-bit, 4.6 GB</sub> | 🟩 401 / **97**<sub>KFlash</sub> | 286 / 62 <sub>MTP</sub> | 281 / 55 <sub>MTP</sub> | 298 / 55 <sub>MTP</sub> | 305 / 42 <sub>MTP</sub> | 308 / 46 | 315 / 38 <sub>MTP</sub> | 308 / 47 | \n| **Qwen3.5-9B** ·`9b`<sub>dense · 4-bit, 9.0 GB</sub> | 🟩 217 / **57**<sub>KFlash</sub> | 159 / 37 <sub>MTP</sub> | — | — | — | 170 / 30 | — | — | \n| **Qwen3.6-35B-A3B** ·`35b:16gb`<sub>MoE, 3B active · 2-bit experts (unsloth's GGUF), 10.8 GB</sub> | 🟩 462 / **52**<sub>KFlash</sub> | × | × | — | × | × | 38 | — | \n| **Nemotron Dory1.0 4B** ·`dory`<sub>looped dense · NVFP4, 3.2 GB</sub> | 🟩 **39**<sub>KFlash</sub> | × | × | × | × | 30 | 4 | × | \n| **Nemotron 3 Nano 4B** ·`nano-4b`<sub>hybrid Mamba-2 · FP8, 5.3 GB</sub> | 🟩 **37**<sub>KFlash</sub> | — | — | — | — | 21 <sub>BF16</sub> | 28 <sub>Q4_K_M</sub> | — | \n| **Nemotron Nano 9B v2** ·`nano-9b`<sub>hybrid Mamba-2 · NVFP4, 7.9 GB</sub> | 🟩 **25**<sub>KFlash</sub> | — | — | — | — | 18 | 14 <sub>Q4_K_M</sub> | — | \n\n**Recursive Flint**\n\n| Model | **Kapitsa** | <sub>llama.cpp d772569</sub> | \n|---|---|---|\n| **Recursive Flint 40B-A2.5B** ·`flint:16gb`<sub>MoE, 2.5B active · 2-bit experts (unsloth's GGUF), 11.9 GB</sub> | 🟩 414 / **61**<sub>KFlash</sub> | 43 | \n\n<sub>Other precisions: `--model 35b:6bit` is five times closer to bf16 than the 4-bit default ([KL](https://github.com/nace-ai/kapitsa/blob/main/docs/benchmarks.md#quality-qwen36-35b-a3b-against-bf16-2026-10-19)). All but Lightning, Dory, the two Nemotron Nanos, Flint and the 35B's `16gb` read images. Memory: 16 GB for the 4B and 9B models, 32 GB\nfor the 27B, 35B, Lightning, Bolt and Flint, 96 GB for Super-VL, 128 GB for Flash-Next. On a 16 or 24 GB Mac, `35b` and `flint` take their `16gb` variants: unsloth's 2-bit GGUFs, downloaded and imported in one step ([how](https://github.com/nace-ai/kapitsa/blob/main/docs/usage.md#gguf-files)). [Every model's variants and draft models](https://github.com/nace-ai/kapitsa/blob/main/docs/usage.md#models-and-downloads)</sub>\n\nKapitsa decodes with **KFlash**. A draft model such as DFlash2 guesses the next seven tokens, with a few candidates for each one. Engines that run it as published (Splash, oMLX, mlx-serve) check one chain of guesses and stop at the first wrong one. KFlash checks a best-first tree of the drafter's candidates in the same pass, so where the drafter hesitates, its runner-ups are checked too and more tokens survive each pass. The tree's size comes from what a verify row costs on your Mac, measured there, and it changes no output: the big model checks every token. It runs on the published drafters as they are: z-lab's DFlash, DFlash2, and NVIDIA's DSpark and LiLiCorr. [How KFlash works](https://github.com/nace-ai/kapitsa/blob/main/docs/kflash.md)\n\n<sub>Watch → Custom → Releases on [nace-ai/kapitsa](https://github.com/nace-ai/kapitsa) puts every new version in your notifications. A drawn illustration of the steps, not a screen recording; the page source is in [tools/readme_images](https://github.com/nace-ai/kapitsa/blob/main/tools/readme_images/README.md).</sub>\n\n[How KFlash works](https://github.com/nace-ai/kapitsa/blob/main/docs/kflash.md) · [Architecture](https://github.com/nace-ai/kapitsa/blob/main/docs/architecture.md) · [Contributing](https://github.com/nace-ai/kapitsa/blob/main/CONTRIBUTING.md) · [Press kit](https://github.com/nace-ai/kapitsa/blob/main/docs/assets/brand/README.md) ·\nApache 2.0, © 2026 [Nace AI](https://nace.ai) · [Third-party notices](https://github.com/nace-ai/kapitsa/blob/main/THIRD_PARTY_NOTICES)", "url": "https://wpnews.pro/news/apple-silicon-inference-engine-kapitsa", "canonical_source": "https://github.com/nace-ai/kapitsa", "published_at": "2026-10-24 07:00:38+00:00", "updated_at": "2026-10-24 07:43:31.101296+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "ai-infrastructure", "developer-tools", "ai-agents"], "entities": ["Kapitsa", "nace-ai", "Apple", "Qwen", "Nemotron", "Recursive", "Flint", "llama.cpp"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/apple-silicon-inference-engine-kapitsa", "markdown": "https://wpnews.pro/news/apple-silicon-inference-engine-kapitsa.md", "text": "https://wpnews.pro/news/apple-silicon-inference-engine-kapitsa.txt", "jsonld": "https://wpnews.pro/news/apple-silicon-inference-engine-kapitsa.jsonld"}}