Show HN: Qwen3.6-35B-A3B on a 16 GB M1 Pro with SSD-streamed MoE

A developer forked the DwarfStar inference engine to enable running large Mixture-of-Experts models like Qwen3.6-35B-A3B on 16 GB Apple Silicon Macs by streaming model weights from SSD. The project explores adaptive memory management and Metal GPU residency policies to push specialized local inference beyond conventional RAM limits.

Specialized local inference for models that do not fit in memory. A transparent research and co-development fork of antirez/ds4 https://github.com/antirez/ds4 , focused on Metal, adaptive SSD streaming, common 16–64 GB Apple Silicon systems, and measured experimentation. Quick start quick-start · Benchmarks measured-results · Models model-status · DSBox https://github.com/andreaborio/dsbox · Upstream diff https://github.com/antirez/ds4/compare/main...andreaborio:ds4:main · Documentation documentation Important This is andreaborio/ds4 https://github.com/andreaborio/ds4 , a fork of . It does https://github.com/antirez/ds4 antirez/ds4 not aim to replace upstream. The goal is to co-develop DwarfStar: explore complementary hardware and model paths here, then propose every general, reproducible improvement back to upstream when it clears the correctness and performance bar. DwarfStar is a small, self-contained inference engine optimized around a narrow set of very large models. It includes native model loading, prompt rendering, tool calling, RAM/on-disk KV state, an HTTP server, a coding agent, GGUF tooling, and correctness and speed tests. It is intentionally not a generic GGUF runner; arbitrary GGUF files are not expected to work. Upstream DwarfStar provides the core engine and leads the high-memory and distributed paths. This fork asks a complementary question: How far can the same specialized design be pushed on the 16–64 GB Macs many developers already own, when the SSD becomes an active model-memory tier? The current work concentrates on: - adaptive Metal residency and routed-expert cache policies across memory tiers; - SSD streaming that accounts for page cache, wired memory, swap, I/O, and throughput together; - safer model-backed experiments near macOS memory limits; - measured GLM 5.2 work and Qwen3.6-35B-A3B bring-up; - GGUF calibration, incremental quantization, and expert-analysis tooling; - keeping useful changes small enough to validate and send upstream. This is primarily a learning and systems-research project. The fork lets work continue while upstream changes are under review; it is not a parallel rewrite or a competing inference ecosystem. The boundary is explicit and reviewable: | Change type | Where it belongs | |---|---| | General, reproducible, backend-safe improvement | Open a PR against antirez/ds4 | This is mandatory, not aspirational: every fork change applicable to an upstream-supported path will be opened upstream once its scope, correctness, and performance evidence are ready. Fork development can continue while that review is in progress. Current upstream work includes 434 https://github.com/antirez/ds4/pull/434 quality-score build fix , 520 https://github.com/antirez/ds4/pull/520 GLM streamed-prefill correctness , and 528 https://github.com/antirez/ds4/pull/528 GLM indexed-prefill prepare . The DeepSeek regression found on the GLM line is tracked in 532 https://github.com/antirez/ds4/issues/532 . See FORK NOTES.md /andreaborio/ds4/blob/main/FORK NOTES.md for the status of each fork change and for sync history. The same policy is part of /andreaborio/ds4/blob/main/MERGE LOG.md MERGE LOG.md . The current /andreaborio/ds4/blob/main/CONTRIBUTING.md CONTRIBUTING.md main delta is always inspectable in GitHub's upstream/fork comparison https://github.com/antirez/ds4/compare/main...andreaborio:ds4:main . Requirements: Apple Silicon, Xcode Command Line Tools, and enough SSD space for the selected model. A 64 GB Mac is the practical reference tier for DeepSeek Flash streaming; the 16 GB path is an experimental low-memory tier, not a speed guarantee. xcode-select --install git clone https://github.com/andreaborio/ds4.git cd ds4 ./download model.sh q2-imatrix make ./ds4 --build-info ./ds4 -m ./ds4flash.gguf --nothink On macOS, AUTO residency keeps the model resident when it safely fits. Otherwise it selects SSD streaming and derives an expert-cache budget from the model geometry and live host memory. Force the SSD path only when you need a controlled run: ./ds4 -m ./ds4flash.gguf --ssd-streaming --ctx 32768 --nothink Start the local API with: ./ds4-server -m ./ds4flash.gguf --ctx 32768 php flowchart LR GGUF "Model GGUF on SSD" -- AUTO "AUTO memory planner" AUTO -- |"safe fit"| RES "Resident model" AUTO -- |"model exceeds budget"| STREAM "SSD-streamed model" STREAM -- FIXED "Mapped fixed / non-routed state" STREAM -- CACHE "Adaptive routed-expert cache" GGUF -- |"cache miss"| CACHE FIXED -- METAL "Metal graph" CACHE -- METAL METAL -- TOKEN "Next token" The fixed model state, KV cache, graph scratch, and macOS file-backed cache all need headroom. The routed-expert cache is the variable tier; making it larger can help only until it starts displacing the pages and allocations the rest of the runtime needs. | Model | Location | Status | Current focus | |---|---|---|---| | DeepSeek V4 Flash | main | Primary supported path | Metal, adaptive SSD streaming, 16–64 GB measurements | | DeepSeek V4 PRO | main | Supported upstream path | High-memory and distributed inference | | GLM 5.2 | codex/glm52-upstream-clean-bench | Experimental branch | Correct streamed prefill and Metal performance on 64 GB | Qwen3.6-35B-A3B qwen35moe | main | Supported opt-in Metal path, model-backed measured | Metal AUTO mapping, live-pressure fallback, strict SSD cache, resident prefill, and parallel resident decode | DS4 includes the experimental, self-describing ds4.expert major.v2 layout for DeepSeek V4. It stores each layer as adjacent gate/up/down expert records without requantizing and without keeping a second routed-weight copy. Canonical GGUFs remain byte-for-byte compatible with every existing backend; native v2 files currently require a complete local Apple Metal model and fail early elsewhere. Conversion, full byte-level verification, compatibility limits, and the model-backed promotion gate are in docs/deepseek-expert-major-v2.md /andreaborio/ds4/blob/main/docs/deepseek-expert-major-v2.md . Until a dated canonical/native benchmark gate is complete, the canonical DeepSeek GGUF remains the release reference. The first M5 Pro SSD tranche is recorded in . The distinctly named experimental artifact is /andreaborio/ds4/blob/main/docs/benchmarks/2026-07-17-deepseek-native-expert-major.md docs/benchmarks/2026-07-17-deepseek-native-expert-major.md , with full conversion provenance and compatibility limits in its model card. https://huggingface.co/andreaborio/DeepSeek-V4-Flash-DS4-ExpertMajor-v2-GGUF DeepSeek-V4-Flash-DS4-ExpertMajor-v2-GGUF The main branch is qualified and measured with one normalized text-only artifact. The recommended release download is the single-layout Qwen3.6-35B-A3B-DS4-ExpertMajor-v1-Q4 K S.gguf from andreaborio/Qwen3.6-35B-A3B-DS4-ExpertMajor-v1-GGUF https://huggingface.co/andreaborio/Qwen3.6-35B-A3B-DS4-ExpertMajor-v1-GGUF . It stores routed weights once in DS4's expert-major order and activates automatically; no sidecar variables are needed. The canonical Qwen3.6-35B-A3B-ds4-Q4 K S.gguf remains supported during migration. The release artifact is 20,808,970,240 bytes only 406,816 bytes larger than the canonical input with SHA-256 fb2b344d49f0c3dfd854cfc11d92ffc873cc93a1d30bf4664e5aea6f1bfef839 .This is not generic Qwen or arbitrary community-GGUF support. The literal environment guard is the experimental opt-in; Metal, power 100, and AUTO residency are the Apple defaults, but are shown below for reproducibility: DS4 QWEN EXPERIMENTAL METAL=1 ./ds4 \ -m /absolute/path/to/Qwen3.6-35B-A3B-DS4-ExpertMajor-v1-Q4 K S.gguf \ --metal --power 100 --ctx 8192 --nothink ds4.expert major.v1 is an explicit DS4 GGUF extension. Other loaders must reject this artifact unless they implement the layout; use the canonical file for llama.cpp, MLX, or other runtimes. Format details, conversion commands, compatibility boundaries, and measured parity are in docs/qwen-expert-major-store.md /andreaborio/ds4/blob/main/docs/qwen-expert-major-store.md . Qwen AUTO selects the full-model mapped Metal mode only when both the fixed Metal working-set budget and a point-in-time host-memory pressure check pass. Under pressure it falls back to SSD and lazily grows the routed-expert cache to the largest complete routing tier admitted by the current conservative snapshot. Above 16 GiB the planner independently reserves the 2.50 GiB static page set, context/runtime memory, and system headroom. On a 16 GiB Mac, AUTO keeps the complete static charge but lets those unpinned, pageable GGUF pages share system headroom. It selects the largest complete 320-expert cache cycle admitted by the remaining live and platform budgets rather than imposing a fixed low-RAM floor. Bounded file-backed inactive pages receive full credit only while macOS reports normal pressure; unknown or elevated pressure retains half-credit and fails closed near the boundary. --resident fails unless both admission checks pass; because pressure can change after the snapshot, this is a conservative admission policy rather than a future-memory guarantee. --ssd-streaming remains the reproducible forced-streaming override. In SSD mode Qwen grows its Metal expert cache in 321-expert slabs about 0.529 GiB instead of taking the generic 4 GiB first slab. Here resident means that DS4 maps the complete tensor payload, disables its explicit SSD expert cache, and executes full-tensor Metal kernels. Metal's residency request is a budgeting hint: it neither pre-faults every GGUF page nor proves that every page remains physically resident as later pressure changes. That stronger physical-residency claim requires separate runtime measurement. All neural math in the supported Qwen path is on Metal. The CPU still performs tokenization, sampling, route readback, cache bookkeeping, and streamed GGUF I/O; a CPU+GPU split of layers or experts is not implemented in this path. The hard SSD cache floor is 321 complete routed experts about 0.53 GiB ; 640 about 1.06 GiB is a useful controlled small-cache tier. Startup and the per-layer path fail closed if the effective locked cache falls below the floor. The runtime has completed model-backed resident and SSD generation on an M5 Pro with 64 GiB, plus bounded SSD generation on a physical M1 Pro with 16 GiB. On production main bd62a0b , AUTO started with 321 cached experts for prefill and grew toward 2,241 for decode. One cold request completed at 10.56/8.24 prefill/generation t/s; four subsequent distinct short prompts had medians of 15.04/9.77 t/s, with normal memory pressure and no new swapouts. This recheck used the canonical migration GGUF; the native ExpertMajor v1 artifact was not copied to the 16 GiB host because only 3.6 GB of disk space was free. The older 4.06/7.03 result remains a conservative 321-expert compatibility floor, not the current production speed. See tests/qwen/README.md /andreaborio/ds4/blob/main/tests/qwen/README.md for the exact artifact contract, reproducible evidence, and current limitations. Metal on Apple Silicon is the current proving ground for fork-specific optimization. The inherited CUDA/DGX Spark and ROCm/Strix Halo DeepSeek paths remain supported targets, but a Metal result is not advertised as a Blackwell or Strix Halo result until it is re-measured on that backend. Best retained local results so far. These rows are not cross-model rankings: each model uses a different artifact, context, and runtime path. | Model | Best measured setup | Prefill | Generation / decode | Status | |---|---|---|---|---| | Qwen3.6-35B-A3B Q4 K S, 20.81 GB | M5 Pro 64 GB, Metal resident | 258.08 t/s | 57.81 t/s | Controlled DS4 prefill A/B, +23.3% over the previous dispatch; greedy output identical | | Qwen3.6-35B-A3B Q4 K S, 20.81 GB | M5 Pro 64 GB, page-touched resident CLI | 218.30 t/s | 63.94 t/s | Best retained real CLI generation number; same rendered prompt and visible continuation as the llama.cpp reference | | Qwen3.6-35B-A3B Q4 K S, 20.81 GB | M1 Pro 16 GB, Metal AUTO to SSD, canonical migration GGUF | 15.04 t/s | 9.77 t/s | Warm median over four distinct short prompts after one cold run; normal pressure, no new swapouts | | DeepSeek V4 Flash IQ2XXS, 86.72 GB | M5 Pro 64 GB, Metal SSD streaming | 20.75 t/s | 12.58 t/s | Direct upstream/fork A/B showed parity, not a fork speedup | | GLM 5.2 ds4-native GGUF, 244.14 GiB | M5 Pro 64 GB, Metal SSD streaming | 9.15 t/s | 0.91 t/s | Indexed-prefill prepare A/B; big prefill win, no decode win | DeepSeek hardware reference bests from the standard speed-bench sweep: | Host | Model | Prefill | Generation | |---|---|---|---| | MacBook Pro M5 Max, 128 GB | Flash q2, 11,707-token context | 463.44 t/s | 25.90 t/s | | Mac Studio M3 Ultra, 512 GB | Flash q2, 11,709-token context | 468.03 t/s | 27.39 t/s | | Mac Studio M3 Ultra, 512 GB | PRO q2, 32,768-token context | 138.82 t/s | 9.56 t/s | | DGX Spark GB10, 128 GB | Flash q2, 7,047-token context | 343.81 t/s | 13.75 t/s | Full commands, samples, and caveats are in docs/benchmarks/2026-07-15-qwen-ds4-vs-llamacpp.md /andreaborio/ds4/blob/main/docs/benchmarks/2026-07-15-qwen-ds4-vs-llamacpp.md , , /andreaborio/ds4/blob/main/docs/benchmarks/2026-07-14-m5-pro.md docs/benchmarks/2026-07-14-m5-pro.md , and /andreaborio/ds4/blob/main/SSD STREAMING VERIFICATION.md SSD STREAMING VERIFICATION.md . /andreaborio/ds4/blob/main/docs/ENGINE REFERENCE.md docs/ENGINE REFERENCE.md More expert-cache RAM is not automatically faster. On memory-constrained Macs, an oversized cache can evict the file-backed pages SSD streaming needs and make decode slower even when Activity Monitor appears to show free memory. AUTO therefore treats the routed-expert cache as variable and preserves headroom for fixed weights, KV, scratch, Metal allocations, and the macOS page cache. During development, a model-backed test bypassed SSD streaming and attempted to make an 80.76 GiB GGUF resident with a 100,000-token context on a 64 GiB Mac. Global wired memory reached roughly 61.36 GiB before a watchdog kernel panic. Crashing the host is not an acceptable test outcome. Current main includes hardware-aware AUTO residency, fail-closed cache admission, bounded benchmark guards, and GPU cleanup before model mappings are released 1523b26 . A stricter guard that rejects resident mappings larger than 90% of physical RAM is tested and published on fix/refuse-oversized-resident-maps at 06fd005 , but is not yet on main . Until it is merged, it must not be described as a mainline guarantee. DSBox https://github.com/andreaborio/dsbox is the companion desktop interface, inspired by Unsloth Studio: discover compatible models, manage ds4, chat locally, connect coding agents, and observe memory, swap, disk, and token throughput without hand-assembling every command. DSBox is a separate project and still a work in progress. https://github.com/andreaborio/dsbox DSBox is an optional companion UI, maintained in a separate repository. : complete model, runtime, server, agent, KV-cache, distributed, backend, and debugging guide. docs/ENGINE REFERENCE.md : experimental Qwen artifact contract, oracle procedure, Metal + SSD commands, measurements, and limits. tests/qwen/README.md : DS4-native GGUF layout, transactional converter, compatibility, and parity evidence. docs/qwen-expert-major-store.md : generic expert-major manifest and separate DeepSeek/GLM qualification plan. docs/expert-major-v2-roadmap.md : upstream-first contribution policy and correctness/performance gates. CONTRIBUTING.md : fork delta and upstreamability ledger. FORK NOTES.md : upstream synchronization history. MERGE LOG.md : Metal build identity, AUTO residency, and benchmark promotion gates. GOLD METAL SSD.md : independent SSD-streaming verification campaign. SSD STREAMING VERIFICATION.md : live, privacy-preserving imatrix collection. ONEDGE IMATRIX.md : mixed-precision expert streaming design and validation. STREAMING MIXED PRECISION.md : expert profiling and prune-mask research. EXPERT PRUNE.md : GGUF, imatrix, quantization, and quality tooling. gguf-tools/README.md Detailed fork additions and research notes The sections below preserve the longer design notes for the fork's research features. They are not an exhaustive commit count: adaptive residency, cache hardening, benchmark guardrails, telemetry, and safe Metal teardown have also evolved since the original five-feature summary was written. The authoritative per-change ledger is FORK NOTES.md /andreaborio/ds4/blob/main/FORK NOTES.md ; upstream syncs are recorded in . /andreaborio/ds4/blob/main/MERGE LOG.md MERGE LOG.md The fork also carries a GLM 5.2 line on codex/glm52-upstream-clean-bench https://github.com/andreaborio/ds4/tree/codex/glm52-upstream-clean-bench : upstream's glm5.2 branch bd89932 plus eleven commits — the streaming prefill correctness fixes proposed as antirez/ds4 520 https://github.com/antirez/ds4/pull/520 real-size prompts were failing under --ssd-streaming ; independently validated by a third party on an M4 Max 128 GB , the indexed-prefill layer-prepare overlap proposed as antirez/ds4 528 https://github.com/antirez/ds4/pull/528 measured prefill ×1.6-2.0 across a 2048-8192 sweep in the PR, ×2.4-2.5 re-measured on short prompts, decode unchanged, greedy output byte-identical , the ds4-native GLM 5.2 GGUF layout support the line runs on, a copy of the RAM guard upstreamed separately, see , and a set of default-off streaming experiments router-ahead prefetch, expert prune/profile hooks, virtual resident decode layers . The short-prompt speedup, the regression below and the MTP gate were re-verified independently with paired A/B runs /andreaborio/ds4/blob/main/FORK NOTES.md FORK NOTES.md ; the sweep figures are from /andreaborio/ds4/blob/main/SSD STREAMING VERIFICATION.md SSD STREAMING VERIFICATION.md 528 https://github.com/antirez/ds4/pull/528 's benchmark. Two caveats, both measured: Upstream's whole DeepSeek-V4-Flash IQ2XXS: 7-8 → ~2-3 tok/s on an M5 Pro 64 GB under glm5.2 line decodes DeepSeek Flash ~2.8× slower than main --ssd-streaming , first token ~5-7 s; bisected to the first commit of the line, verified twice on separate days . Keep DeepSeek work on main ; reported upstream as antirez/ds4 532 https://github.com/antirez/ds4/issues/532 . Speculative decode MTP on streamed GLM is a measured NO-GO : the blk.78 nextn acceptance probe branch, a reusable measurement tool; GLM 5.2 ds4-native build reads ~55% acceptance against the ~75% needed to pay for the extra I/O. feat/glm-mtp-probe The older bring-up branch wip/glm52-metal64-strict-probe https://github.com/andreaborio/ds4/tree/wip/glm52-metal64-strict-probe predates this line and is kept as history. Upstream collects the routed-MoE importance matrix imatrix offline from a fixed corpus ds4 --imatrix-dataset … --imatrix-out … . This fork lets ds4-server collect it from the live prompt stream on the device , so a quantized model can be re-calibrated to its actual workload , without ever storing a single user prompt. The only artifact is the imatrix: aggregate per- layer, expert activation statistics squared activations + hit counts , a structure that cannot hold prompt text. ds4-server -m model.gguf --imatrix-out edge.dat collect from live traffic ds4-server -m model.gguf --imatrix-out edge.dat --imatrix-every 128 --imatrix-min-requests 32 Default off zero behavioral change ; opt-in via --imatrix-out , with periodic snapshots --imatrix-every and a minimum-requests guard --imatrix-min-requests . Full design, wiring, limits and privacy verification in ONEDGE IMATRIX.md /andreaborio/ds4/blob/main/ONEDGE IMATRIX.md . Re-forging a variant say, adding a per-layer Q4 "boost" on top of an IQ2 build that used the same imatrix normally regenerates every routed-expert tensor from the FP weights, even the ones that don't change. But quantization is deterministic in FP weights, target type, imatrix slice , so an unchanged tensor is byte-identical to the one already sitting in a prior build. Recomputing it is pure waste. --reuse PRIOR.gguf copies a planned output tensor straight from PRIOR when its name, target type and shape match, and quantizes only the tensors that actually changed the boosted layers, at their new type . 1. build the 2-bit base once gguf-tools/deepseek4-quantize --hf FP --template base.gguf --imatrix coder.dat \ --out coder-iq2.gguf 2. every boost variant reuses the base's unchanged layers, re-quantizing only the boosted ones gguf-tools/deepseek4-quantize --hf FP --template base.gguf --imatrix coder.dat \ --reuse coder-iq2.gguf \ --tensor-type blk.30.ffn gate exps.weight=q4 k … --out coder-q4boost.gguf Measured DeepSeek-V4-Flash, a 6-of-43-layer Q4 boost over an IQ2 base : a full build is ~80 minutes; the same variant via --reuse took 5.5 minutes 1,310 of 1,328 tensors copied, 18 regenerated , about a 14× speedup . The output was verified byte-for-byte identical to a from-scratch build across all 1,328 tensors. The fast build is not an approximation, it is the same file. Correctness. Every build stamps a quantize.reuse key GGUF KV: an fnv1a64 over the safetensors index, each weight shard's size and mtime, the imatrix content, and a template structural salt. --reuse copies a tensor only when PRIOR's key matches this build and the per-tensor type and shape match, so a boosted tensor different target type is regenerated, and a stale / foreign / keyless prior changed weights, imatrix, or recipe safely falls back to a full quantize. Copied bytes are size-checked against the plan a hard error on any mismatch , and --reuse refuses to alias --out . This is not present in llama.cpp, which always requantizes from the source weights; the closest prior art is splicing GGUF tensors by hand. Changing the imatrix only changes the tensors the imatrix actually steers the routed expert families: the importance vectors re-allocate bits inside those tensors . Everything else — attention, shared experts, norms, embeddings, output — is byte-identical across builds that share the same FP weights and template. So every build now also stamps quantize.reuse key weights : the same fnv1a64 without the imatrix folded in. When PRIOR matches the full key, behavior is unchanged; when it matches only the weights key same weights, different imatrix — the re-calibration case , --reuse copies the imatrix-independent tensors and regenerates only the steered ones: reuse: PRIOR.gguf shares the weights key … but not the imatrix — copying imatrix-independent tensors, regenerating the steered ones The dependence test is conservative and mirrors the generators' own imatrix lookups routed exps. families always count as steered; regular tensors are probed with the exact same name resolution generate regular uses , so over-approximation can only cost an unneeded regeneration, never a stale byte. Priors built before this change carry only the old key and keep the old all-or-nothing behavior. Measured DeepSeek-V4-Flash, 1,328 tensors, M5 Pro : a full re-calibration — same recipe, coder.dat → general.dat — copied 1,199 of 1,328 tensors and regenerated the 129 routed-expert tensors with the new imatrix, in ~45 minutes vs ~80 for the full quantize . Byte-level verification: 40/40 sampled imatrix-independent tensors identical to the prior, 16/16 sampled expert tensors changed, tensor tables identical. The change went through an adversarial 3-lens review that rejected the first cut two stale-byte paths, one strict-mode abort — all reachable, all fixed before this exercise: the no-imatrix gate, the coverage fingerprint, the I32 probe exclusion . Upstream --ssd-streaming assumes routed-expert tensors are quantized uniformly across layers. A GGUF with a few layers boosted to Q4 K over an IQ2 base the forgequant boost recipe failed every request under streaming model range … is not covered by mapped model views while serving fine with full residency. Two compounding uniformity assumptions are fixed: the streaming prefill span set now also maps the exps tensors of off-class "boosted" layers, so they are read through mmap'd no-copy views; and the single-size-class expert cache pre-seeds its slab size at startup and rejects off-size layers which use the mapped path instead of silently adopting their size and corrupting the slot accounting. Uniform models are verified byte-identical under the change 3/3 builds , full-residency paths are untouched, and mixed models were validated with the canary benchmark plus entire eval suites. Full diagnosis, design and behavior guarantees in STREAMING MIXED PRECISION.md /andreaborio/ds4/blob/main/STREAMING MIXED PRECISION.md ; reported upstream with diagnosis and workaround in antirez/ds4 388 https://github.com/antirez/ds4/issues/388 . Update upstream converged : antirez has since implemented equivalent mixed-precision streaming upstream. After the latest sync this fork takes upstream's implementation of weights streaming layer experts uniform the only merge conflict; the two designs converged — see MERGE LOG.md /andreaborio/ds4/blob/main/MERGE LOG.md . This addition is effectively now upstream. Two small, opt-in hooks for studying which experts a domain actually needs, used by the forgequant layer/expert A/B work: — the expert profiler DS4 EXPERT PROFILE FULL ds4 expert profile write layer emits the full per-expert ranking instead of the top-16, so a static prune/keep set can be chosen per layer from real routing statistics.— point it at a DS4 EXPERT PRUNE MASK 43 × N EXPERT grid of '0'/'1' '1' = prune . The mask is applied to the CPU router's probs before top-k masked experts get a large-negative sentinel so they never win , letting each token route to its next-best surviving expert. This measures "how much of the domain lives in a few experts" without re-quantizing anything. the mask lives in the CPU router, so enable it streaming-IQ2 path , then prune: DS4 METAL ENABLE STREAMING IQ2 CPU ROUTER=1 DS4 EXPERT PRUNE MASK=mask.txt \ ds4 -m coder-iq2.gguf -p "…" --ssd-streaming - "ds4: expert prune mask ACTIVE N experts pruned from mask.txt" Both default off zero behavioral change . The mask affects only routed non-hash layers, and only when the CPU router is active streaming-IQ2 or PRO-Q4 paths . Details in EXPERT PRUNE.md /andreaborio/ds4/blob/main/EXPERT PRUNE.md . The long-form guide now lives in docs/ENGINE REFERENCE.md /andreaborio/ds4/blob/main/docs/ENGINE REFERENCE.md . It covers model downloads, full-resident and SSD-streamed operation, distributed inference, power controls, the native agent, benchmarking, capability evaluation, CLI, server/tool calling, disk KV cache, backends, steering, test vectors, and debugging. Keeping the manual separate makes this README a reviewable landing page while preserving the full operational reference. DwarfStar is beta software and ds4-agent remains alpha. The core engine and upstream direction come from antirez/ds4 https://github.com/antirez/ds4 . The project also exists thanks to the kernels, formats, and engineering work of and GGML. https://github.com/ggml-org/llama.cpp llama.cpp Released under the MIT license /andreaborio/ds4/blob/main/LICENSE . Contributions follow the upstream-first policy /andreaborio/ds4/blob/main/CONTRIBUTING.md .