Captured 2026-09-02. This is a redacted reconstruction reference: it records explicit local settings, storage paths, installed-model metadata, and profiles. It intentionally omits credentials and request statistics.
System settings
Server and lifecycle
Setting
Value
Bind address / port
127.0.0.1:8000
Log level
info
CORS origins
Server aliases
localhost, 127.0.0.1, two private-LAN addresses, 10.42.0.0, 10.43.0.0
network: HTTP/HTTPS/no-proxy and CA-bundle fields empty. claude_code: mode cloud; model fields null.
integrations: Codex, OpenCode, OpenClaw, Hermes, Pi, and Copilot models null; OpenClaw tools profile coding. MarkItDown enabled but not exposed as a model, 25 MB/file, five files/request, engine markitdown. DDGS search with three results, snippet mode, 20,000 characters.
ui.language: en; idle timeout null. Authentication material is configured locally and omitted here. API-key verification is skipped. Brave search key is empty.
GUI/app-support preferences
Setting
Value
Base path
~/.omlx
Port
8000
Model directory
~/.lmstudio/models
Launch at login
false
Start server on launch
false
Shared model settings
These values are stored for every configured model unless a model/profile table explicitly gives a different value.
Here is everything the knowledge base holds on your oMLX + Qwen setup, as of 2026-09-03.
The rig
M5 Max Mac, 128GB unified memory. oMLX (omlx.ai) serving an OpenAI/Anthropic-compatible API on localhost:8000. Client is OpenCode, used for code and log/text analysis only, never images.
Orchestrator / daily driver: lmstudio-community/Qwen3.8-27B-MLX-6bit, ~20.6GB loaded. Qwen3.8-27B is a VLM with hybrid attention (gated DeltaNet linear + gated full attention, arch tag qwen35). The 6-bit build engages oMLX's custom Metal kernel: omlx.custom_kernels.qwen35_prefill.fast - Qwen qmm NAX dispatch enabled.
Worker: EigenLabs/Qwen3.6-35B-A3B-MLX-mixed-4bit, 35B total / ~3B active, ~20GB. Mixed quant protects shared experts, attention/DeltaNet projections, embeddings and output at 5-6bit/BF16, which is why tool-call JSON stays reliable.
Sampling (settled 2026-08-19): temperature 0.5, top_p 0.95, top_k 20, min_p 0, presence_penalty 0, repeat_penalty 1.0. enable_thinking true with chat_template_kwargs {"reasoning_effort": "low"}. preserve_thinking left true, since false mutates the prefix and kills the cache. Context 150k in oMLX. max_tokens 4096, set in OpenCode, not oMLX, because OpenCode overrides the oMLX per-model value.
Measured performance
27B 6-bit: 15.5-17.8 tok/s decode, ~487-600 tok/s prefill, sagging toward ~400 at long context.
35B-A3B 4-bit worker (measured on your machine 2026-09-03): 82-123 tok/s decode, ~1,450 tok/s prefill, and perfect prefix reuse on consecutive turns (45,056 of 45,056, then 49,152 of 49,152). That confirms the ~7.5x decode advantage previously predicted from MoE sparsity analysis.
Cold start at ~131k context is 5+ minutes. Not a regression, just 131,072 tokens at ~487 tok/s plus quadratic sag.
oMLX logs report tok/s as decode-only while the accompanying wall-clock includes prefill.
Model verdicts
Rejected avlp12/Qwen3.8-27B-Alis-MLX-8bit (27.9GB, 333 vision tensors + MTP head): 14.6-15.1 tok/s at 62-66k vs the 6-bit's 17.0-17.8 at 78k, and the MTP head never fired. MLX 6-bit beats 8-bit on this architecture; 8-bit vs BF16 perplexity differs by only ~0.14%.
Ruled out DeepSeek-V4-Flash: 103GB resident, cannot coexist with the orchestrator, only 33 tok/s.
Qwen3.8-Flash-Next-oQ4e: third-party benchmark only, unknown hardware. Wins prefill 1.65-2.15x with the gap widening, but decode crosses under the 27B past ~16k and it costs ~28GB more peak. The sweep stops at 32k, far short of your working context.
Quality tradeoff accepted: SWE-bench Verified 73.4 (35B-A3B) vs 77.2 (27B dense). Roughly 4 points for 7.5x speed, which is why the orchestrator stays dense.
MTP and quantization
MTP gates: real mtp.* weights in the checkpoint (metadata alone is silently skipped), process-wide attach flag on, supported arch, batch size 1. Verify by log line, never the model card.
Depth 1 only. Depth 2 and 3 give nothing, because single-stream decode is ~95% backbone-compute-bound. Same reason Q8 to Q4 only buys ~1.4x.
MTP degrades with batching, so parallel subagent fan-out trades it away. On the A3B it is largely moot anyway (+12% vs +75% on the dense 27B).
8-bit KV cache: no help. Prefill is compute-bound, K is quantization-sensitive, and it risks dropping off the custom Metal kernel. MTP is decode-only and cannot touch initial load.
Prefix cache (the big lever) Blocks are 256 tokens, content-addressed by SHA-256 chain hash, stored as .safetensors under ~/.omlx/cache. Index is rebuilt on startup, so restart survival is intended. Sessions sharing a system prompt and tool schemas share blocks automatically.
oMLX auto-enlarges block size 256 to 2048 for ArraysCache hybrid models (both your models), overriding any manual smaller value. The old advice to try 512 does not apply.
boundary_snapshot_unavailable skips are benign on hybrid models. With block_size 2048 the cache advances in a 2048-token staircase; the worst case loss is the trailing partial block. Do not chase those log lines.
Gotchas: the hot RAM tier is write-back and only flushes on clean shutdown, so a kill -9 loses everything (and on 128GB, memory pressure may never trigger a flush). One differing byte in the first 256 tokens invalidates all downstream blocks, so static content goes at the head and date/cwd/git-branch/session-id at the tail. hot_cache_only disables SSD persistence entirely. Enabling TurboQuant invalidates the entire on-disk cache.
Diagnostic fork via /admin stats: saves ~0 means write-path failure (unclean shutdown); saves high but loads ~0 means restore rejection.
Root cause of your slow loads (2026-08-31): prompts diverge from cache at ~token 4332, inside the OpenCode tool-schema block. Prime suspect is nondeterministic tool ordering from MCP servers, then date/cwd/git in the system prompt, then a file tree. Fix is to diff two tokenized prompts around 4200-4500 and move all session-varying content to the tail. Capacity is not the constraint (evictions are 0), so raising hot_cache_max_size to 40-48GB is second priority.
compaction.prune: true in OpenCode is a prefix-cache killer and should be off. It rewrites the middle of the conversation and forces a full re-prefill, roughly 3 minutes at 100k. You had it on as of 2026-09-03, alongside preserve_recent_tokens: 8192 and reserved: 10000. General rule: compacting less often beats keeping context small.
Where settings live
Per-model settings: /admin dashboard, persisted to model_settings.json in the oMLX base path. Not ~/.omlx/settings.json, which is server/admin config.
No dedicated reasoning-level control. For Qwen3.8, set reasoning_effort (xhigh/medium/low) inside chat_template_kwargs, and do not also set enable_thinking in both places.
Profiles are named setting bundles exposed as separate model IDs (e.g. qwen3.8-27b:think-low). They share the base model's memory and apply per request without a reload, so they are the way to A/B without paying a cold prefill.
Config precedence: defaults, then ~/.omlx/settings.json, then OMLX_* env, then CLI. Most cache settings need a restart.
OpenCode side
Agent architecture at github.com/Gipetto/opencode-config: orchestrator and plan on the 27B 6-bit (edit denied), explore/implement/verify on the A3B, review back on the 27B. subagent_depth 1, small_model and compaction model on the A3B. Verification is split so the A3B does mechanical checks (~7s) and the 27B only judges the green path (~25s).
Context overflow was solved 2026-08-28: base system prompt was 64,193 tokens, of which ~55-58k was tool JSON from ~146 tools across 7 MCP servers. Fix was top-level tools with server-prefixed globs set to false, re-enabled per agent. permission: deny does not reclaim MCP context. After the fix: orchestrator ~60-70k, explore ~18-22k, implement/verify ~14-16k.
Corollary that reframes everything: your 62-80k "typical working prompts" were mostly tool schemas, with only 0-15k of actual content. Trimming tools cuts prefill linearly.
Tool-call churn was resolved 2026-08-19. Cause not definitively attributed among repeat_penalty 1.1 to 1.0, max_tokens 8192 to 4096, reverting to 6-bit, and adding the kindex policy plugin.
Known contradiction found 2026-09-03: the shared AGENTS.md was written for a single do-everything agent and fights the role split, both on "prefer editing files directly" and on whether the orchestrator may consult kindex.
Open items
Whether oMLX holds two models resident concurrently or swaps on demand. Still listed as an unverified blocker for automated delegation, though the 2026-09-03 A3B measurements suggest the routing is running in practice.
Whether a Qwen3.8-generation A3B exists with MLX 4-bit quants. All verified A3B builds are 3.6 series, and mixed generations mean different tokenizers, ruling out shared-drafter tricks.
Whether the model reliably consults kindex before grepping.
Compaction settings should be retuned now that the orchestrator base prompt has dropped.