{"slug": "apertura-a-from-scratch-objective-c-mlx-rebuild-of-gemma-4-for-apple-silicon", "title": "Apertura A from-scratch Objective-C++ / MLX rebuild of Gemma-4 for Apple Silicon", "summary": "Apertura, a from-scratch Objective-C++/MLX rebuild of Google's Gemma-4 for Apple Silicon, runs the entire Gemma-4 text family natively on a Mac with bit-exact conformance to the PyTorch reference, achieving 80/80 and 89/89 matching tokens on real prompts before bf16 near-ties diverge. The open-source project, developed against MLX 0.31.2, supports all four architectures (31B dense, 26B MoE, E2B/E4B elastic, 31B QAT) selectable via config.json, and is designed as an inspectable research instrument with a conformance trace and CLI driver for sweeping experts, coarsening precision, and toggling reasoning.", "body_md": "**A from-scratch Objective-C++ / MLX rebuild of Google's Gemma-4 for Apple Silicon — built to be inspected, observed, and experimented with.**\n\nMost language models are run behind glass: you send a prompt and get an answer, with no way to watch the machine think. Apertura is the opposite. It's a faithful, class-for-class, op-for-op re-implementation of the HuggingFace PyTorch `Gemma4TextForCausalLM`\n\n, written in readable Objective-C++ on top of [MLX](https://github.com/ml-explore/mlx), that runs the whole Gemma-4 text family **natively and entirely on a Mac** — no cloud, no Python at inference time. Every layer is a small, inspectable object you can trace, freeze mid-forward, quantize, or take apart.\n\nIt is a research instrument first and a runtime second: correctness is gated by **bit-exact conformance against the PyTorch reference**, not by vibes.\n\n**Faithful.** A clean rewrite from`modeling_gemma4.py`\n\n, conformance-tested op-by-op against PyTorch. Greedy generation matches the reference token-for-token down to the floating-point-order floor (e.g. 80/80 and 89/89 matching tokens on real prompts before the first bf16 near-tie diverges).**Universal.** One codebase plays the**entire Gemma-4 text family**, switched by`config.json`\n\nalone — no code changes.**Local.** Runs offline on Apple Silicon via MLX. A 31B-parameter model holds a conversation on a laptop.**Observable.** A conformance trace exposes the scaled embedding, every decoder-layer output, and the final norm; the CLI driver lets you sweep experts, coarsen precision, and toggle reasoning, then watch what changes.\n\nAll four architectures are verified to match the PyTorch reference (argmax + greedy) at bf16, selectable purely by the model's `config.json`\n\n:\n\n| Model | Architecture | Notes |\n|---|---|---|\n| Gemma-4 31B | Dense | 60 layers, hybrid local/global attention |\n| Gemma-4 26B | Mixture-of-Experts | 128 experts, top-8 routing (dense or sparse path) |\n| Gemma-4 E2B / E4B | Elastic | Per-Layer Embeddings (PLE) + shared-KV layers |\n| Gemma-4 31B QAT | Quantization-aware-trained | runs faithfully at bf16 |\n\nThe faithful Gemma-4 details ported exactly include: hybrid 5:1 local/global attention, dual head_dim (256 local / 512 global), partial RoPE on global layers, QK-norm before RoPE, weightless V-norm with `attention_k_eq_v`\n\n, the 4-norm sandwich + per-layer `layer_scalar`\n\n, tied embeddings, the bf16-rounded embedding scale, and the final-logit softcap (30.0). See [ aptransformer/MoE_REFERENCE.md](/apocryphx/Apertura/blob/main/aptransformer/MoE_REFERENCE.md) for the annotated MoE path and config switches.\n\n```\naptransformer/         The MLX compute framework (the model itself)\n  ESModelConfig        parses config.json — one config drives the whole family\n  ESWeightLoader       sharded safetensors -> mx::array, cast to compute dtype\n  ESEmbedding ESLinear ESRMSNorm ESRotaryEmbedding ESMLPBlock\n  ESAttention ESKVCache ESRouter ESExperts          attention, cache, MoE\n  ESDecoderLayer ESGemma4TextModel ESGemma4TextForCausalLM\n  ESSampler ESGenerationLoop                        sampling + prefill/decode loop\n  ESConformance                                     fixture loading + deviation stats\nAperturaResearch/      Command-line driver\n  main.mm              conformance sweep, generation, benchmarks, expert ladder\n  ESTokenizer          thin wrapper over ObjCTokenizer\n  ESChatTemplate       Gemma-4 chat grammar: roles, reasoning channel, tool calls\naptransformerTests/    XCTest conformance + primitive tests\nTools/                 PyTorch fixture generators (run once, in a torch env)\n```\n\n**macOS on Apple Silicon**(Metal).** MLX**—`brew install mlx`\n\n(developed against 0.31.2). Headers in`/opt/homebrew/include`\n\n, lib`-lmlx`\n\n.— a pure-Objective-C, byte-identical HuggingFace tokenizer, used for encode/decode. Checked out as a sibling directory.[ObjCTokenizer](https://github.com/apocryphx/ObjCTokenizer)**Model weights**— a Gemma-4 HuggingFace snapshot (`config.json`\n\n+ sharded safetensors +`tokenizer.json`\n\n).**(Conformance only)** a Python env with`torch`\n\n+`transformers`\n\nto regenerate fixtures from the reference implementation. Not needed to run inference.\n\n**Xcode** (primary): open `Apertura.xcodeproj`\n\nand build the `aptransformer`\n\nframework, the `AperturaResearch`\n\nCLI target, and the `aptransformerTests`\n\ntest target. The `aptransformer`\n\nfolder is a synchronized group, so new `.mm`\n\nfiles are picked up automatically.\n\n**Direct clang** (research/dev build): with MLX in `/opt/homebrew`\n\nand `ObjCTokenizer`\n\nchecked out as a sibling, compile the framework sources, the driver, the tokenizer wrapper, and the prebuilt ObjCTokenizer objects together:\n\n```\nOCT=../ObjCTokenizer/ObjCTokenizer\nclang++ -std=gnu++20 -fobjc-arc -ObjC++ -O2 \\\n  -I/opt/homebrew/include -Iaptransformer -IAperturaResearch -I\"$OCT/..\" -I\"$OCT\" \\\n  aptransformer/ES*.mm AperturaResearch/main.mm AperturaResearch/ESTokenizer.mm \\\n  AperturaResearch/ESChatTemplate.mm build/oct/*.o \\\n  -L/opt/homebrew/lib -lmlx -licucore \\\n  -framework Foundation -framework Metal -framework Accelerate \\\n  -framework QuartzCore -framework MetalPerformanceShaders \\\n  -o build/AperturaResearch\n```\n\nThe driver takes a model snapshot directory and a mode. A few examples:\n\n```\nSNAP=~/.cache/huggingface/hub/models--google--gemma-4-31b-it/snapshots/<hash>\n\n# Chat (reasoning off): build a Gemma-4 prompt, generate, parse the answer\n./build/AperturaResearch \"$SNAP\" --chat \"Name three primary colors.\" --decode 40\n\n# Chat with the reasoning channel exposed\n./build/AperturaResearch \"$SNAP\" --think --chat \"A bat and ball cost \\$1.10...\" --decode 400\n\n# Quantized inference (4-bit weights, 8-bit embedding) + operator fusion\n./build/AperturaResearch \"$SNAP\" --quant 4 --quant-embed 8 --fused --generate \"...\" 200\n\n# Experiment: sweep an MoE model's active experts 128 -> 4 and watch the output shift\n./build/AperturaResearch \"$SNAP_26B\" --expert-ladder /path/to/prompt_ids.safetensors\n\n# Conformance + throughput\n./build/AperturaResearch \"$SNAP\" /path/to/fixtures.safetensors    # per-op + argmax/greedy gate\n./build/AperturaResearch \"$SNAP\" --bench --prefill 512 --decode 128\n```\n\nKey flags: `--chat`\n\n/ `--system`\n\n/ `--think`\n\n/ `--sample`\n\n, `--quant N`\n\n/ `--quant-embed [N]`\n\n/ `--quant-kv N`\n\n, `--fused`\n\n, `--moe-sparse`\n\n, `--expert-ladder`\n\n, `--generate`\n\n, `--decode`\n\n/ `--prefill`\n\n, `--longctx`\n\n, `--bench`\n\n.\n\n**Quantization**— 4/8-bit weights, independent embedding/LM-head bits, and a quantized KV cache.** Operator fusion**—`mx::fast`\n\nkernels and`mx::compile`\n\nfor RMSNorm, RoPE, SDPA, GeLU.**Sparse MoE routing**—`gather_mm`\n\n/`gather_qmm`\n\nso only the selected experts are computed.**Gemma-4 chat grammar**(`ESChatTemplate`\n\n) — turns/roles, the on/off reasoning channel, and tool-call parsing, built at the token-id level to match the reference exactly.**Sampling**— greedy plus temperature / top-k / top-p.\n\nDecode is memory-bandwidth-bound: bf16 on the 31B runs at roughly the same throughput as llama.cpp, and the quantization + fusion + sparse-MoE levers scale it up substantially. Numbers depend on the machine. As of 2026-07-21 the Q4 engine measures at **llama.cpp parity on both decode and prefill** at practical context lengths (94-99.5% decode, prefill parity through ~10K-token prompts), with no custom Metal kernels — measured standing, per-lever record, and the benchmark methodology (thermal gating, process hygiene) live in [ aptransformer/PERFORMANCE_ROADMAP.md](/apocryphx/Apertura/blob/main/aptransformer/PERFORMANCE_ROADMAP.md).\n\n`Tools/generate_fixtures.py`\n\n(run once in a torch env) captures the reference's intermediate tensors and greedy token sequence. `ESConformance`\n\nloads them and reports per-op deviation (max / median / p99); the acceptance gate is **exact per-position argmax and greedy token-id match**, with numeric tolerances set to the bf16 floor. Cross-engine divergence at near-ties (Metal/MLX vs MPS/PyTorch vs llama.cpp) is the expected floating-point-order floor, not a correctness gap.\n\n- The HuggingFace\n`transformers`\n\nGemma-4 reference (`modeling_gemma4.py`\n\n) — the authoritative oracle. [MLX](https://github.com/ml-explore/mlx)— the Apple Silicon array framework.[ObjCTokenizer](https://github.com/apocryphx/ObjCTokenizer)— the tokenizer.\n\n[MIT](/apocryphx/Apertura/blob/main/LICENSE) © 2026 Kolja Wawrowsky.", "url": "https://wpnews.pro/news/apertura-a-from-scratch-objective-c-mlx-rebuild-of-gemma-4-for-apple-silicon", "canonical_source": "https://github.com/apocryphx/Apertura", "published_at": "2026-08-18 17:57:13+00:00", "updated_at": "2026-08-18 18:11:22.100150+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research", "ai-tools"], "entities": ["Google", "Gemma-4", "MLX", "HuggingFace", "PyTorch", "Apertura", "Objective-C++", "Apple Silicon"], "alternates": {"html": "https://wpnews.pro/news/apertura-a-from-scratch-objective-c-mlx-rebuild-of-gemma-4-for-apple-silicon", "markdown": "https://wpnews.pro/news/apertura-a-from-scratch-objective-c-mlx-rebuild-of-gemma-4-for-apple-silicon.md", "text": "https://wpnews.pro/news/apertura-a-from-scratch-objective-c-mlx-rebuild-of-gemma-4-for-apple-silicon.txt", "jsonld": "https://wpnews.pro/news/apertura-a-from-scratch-objective-c-mlx-rebuild-of-gemma-4-for-apple-silicon.jsonld"}}