Gemma 4 26B in 2GB Is Real. The Headline Is Still Misleading.

TurboFieldfare, an independent Swift and Metal runtime, reports running Google's Gemma 4 26B A4B Mixture-of-Experts model with a resident memory footprint of roughly 1.9–2.1GB on Apple Silicon by streaming routed experts from a 14.3GB SSD installation, though the full system requires at least 8GB of RAM. The runtime achieves 5.10–6.30 decode tokens per second on an 8GB M2 MacBook Air and 31–35 tokens per second on a 24GB M5 Pro, but the project cautions that these are workload-specific measurements, not universal guarantees.

Gemma 4 26B does not have a universal 2GB RAM requirement. TurboFieldfare reports a roughly 1.9–2.1GB footprint on Apple Silicon by keeping a 4K FP16 KV cache and shared model core resident while streaming routed experts from a 14.3GB SSD installation. The result is real, specialized, text-only, and different from conventional local inference. There is a particular kind of benchmark headline that becomes less accurate as it becomes more shareable. “A 26-billion-parameter model runs in 2GB” is one of them. The number comes from TurboFieldfare https://github.com/drumih/turbo-fieldfare , an independent Swift and Metal runtime built for the instruction-tuned Gemma 4 26B A4B model on Apple Silicon. The project’s published measurements support the roughly 2GB footprint. But the implementation does not squeeze the entire model into a tiny in-memory representation. It changes what must stay resident. That distinction turns the result from magic into useful engineering. TurboFieldfare’s 2GB headline covers the resident model weights and a 4K FP16 KV cache in its reference configuration. The complete computer still needs at least 8GB of system memory, and the repacked text-only model occupies approximately 14.3GB on SSD. Gemma 4 26B A4B is a Mixture-of-Experts model. It has approximately 25.2B total parameters but activates approximately 3.8B parameters for each token. Google’s model card describes 128 routed experts, eight selected per token, plus one shared expert. Conventional runtimes keep the complete quantized model resident in RAM or VRAM. TurboFieldfare takes another route: The model has not become smaller. Resident memory has been exchanged for storage reads, cache behavior, and platform-specific engineering. “Gemma 4 26B runs in a specialized 2GB resident-memory configuration” is defensible. “Gemma 4 26B only needs 2GB of RAM” is not. SSD-backed expert streaming is useful because each Gemma 4 token uses only a subset of the routed experts. TurboFieldfare can keep a bounded expert cache and fetch misses instead of retaining every routed expert in memory at once. That approach would be far less useful for a dense model, where every weight participates in every forward pass. The I/O demand would become much harder to hide because each generated token would require the full model rather than a routed subset. The project’s diagnostic measurements make the cost visible. In one instrumented M2 run, expert reads accounted for 83.1 milliseconds per token, while command-buffer waiting accounted for another 55.6 milliseconds. The project explicitly cautions that instrumentation changed performance, so these figures are an explanation of that run — not independent speedups or a general performance bound. The valuable point is simpler: the SSD is part of the inference path, not just the installation location. TurboFieldfare reports 5.10–6.30 decode tokens per second on an 8GB M2 MacBook Air and 31–35 tokens per second on a 24GB M5 Pro. The project labels these as workload-specific measurements rather than universal guarantees. The M2 benchmark rows used fresh processes, a warm but uncontrolled file cache, and prompts ranging from 6 to 1,017 tokens. Decode rate excludes installation, model loading, and prompt prefill. The longest published M2 row reported 36.7 seconds to the first generated token and 5.38 tok/s decode for a 1,017-token prompt. That performance can be useful for private extraction, background summarization, coding assistance, or batch work where a fanless laptop’s latency is acceptable. It is not equivalent to a managed, concurrent API service. Throughput also sets a hard capacity limit. At 5.1 tok/s, a machine can theoretically decode about 13.2 million tokens in a 30-day month if it generates continuously. At 50% utilization, the number falls to about 6.6 million. Producing four million tokens takes approximately 9.1 uninterrupted days before prefill, queueing, failures, restarts, and maintenance. Capacity planning cannot be replaced with a one-time laptop price. No published result combines the roughly 2GB footprint with Gemma 4’s full 256K context. Google’s model capability and TurboFieldfare’s tested local configuration are separate facts. The local runtime currently offers 4K, 8K, 16K, 32K, and 64K context settings. Its default app setting is 4K, and larger FP16 KV caches use more memory. The experimental local server defaults to 16K when launched from the documented command, but that does not preserve the 2GB headline configuration. Long-context buyers should ask two different questions: Collapsing those questions produces a specification that no single configuration demonstrated. Google provides Gemma 4 26B through the Gemini API under the model ID gemma-4-26b-a4b-it. The hosted route supports text and image input, configurable thinking, system instructions, function calling, and multi-turn conversations without requiring local model storage or inference operations. Google currently lists Gemma 4 input, output, and context caching as free on the Gemini API free tier, with no paid Gemma 4 tier listed. That makes a simplistic “local avoids token costs” argument especially weak today. Free-tier access is still not a permanent production contract. Quotas, availability, data-use terms, and future paid options may change. Google states that free-tier content may be used to improve its products, so confidential, regulated, or customer-owned data requires a separate review. The hosted API and local runtime also expose different product surfaces: RequirementOfficial Gemini APITurboFieldfare localFast setupStrongRequires build and ~15GB transferImage inputYesNoOffline operationNoYesData stays on-deviceNoYesFull model contextUp to 256K supported4K–64K runtime settingsBursty trafficManaged, quota-limitedOne local machine’s capacityAuthentication/TLSManagedNot present in loopback serverRuntime controlLimitedHighCurrent direct token feeFree tierNo provider fee; local costs remain Local inference is cheaper only when its complete cost, achievable capacity, and accepted output quality beat the hosted alternative for the actual workload. Suppose a $1,200 Apple Silicon machine is amortized over 24 months. The hardware component is $50 per month. If it reliably produces four million accepted output tokens per month, hardware amortization alone is $12.50 per million output tokens. That number excludes electricity, SSD wear, engineering, monitoring, downtime, failures, retries, review, and the opportunity cost of occupying the machine. It also assumes the workload fits inside the throughput window. A more useful comparison is: Local cost per accepted task= hardware + electricity + storage + engineering + retries + review / accepted tasks Hosted cost per accepted task= token and request charges + retries + review / accepted tasks When the official API has no direct fee inside free-tier limits, privacy and control — not savings — may be the strongest reason to run locally. TurboFieldfare provides an experimental Chat Completions server at http://127.0.0.1:8080/v1, but the project says it has no remote authentication or TLS and should not be exposed through a proxy or tunnel. That warning should be treated as an architectural boundary. Loopback access is useful for local applications and coding tools. Remote or multi-user production service would require a separate, reviewed serving layer with authentication, authorization, TLS, request limits, queueing, supervision, readiness checks, memory and SSD monitoring, privacy-aware logs, overload behavior, and failover. The local server can return model-generated tool calls, but the client must inspect, authorize, and execute them. A generated function name is not permission to perform an action. As of July 30, 2026, CometAPI does not list Gemma 4 26B. The documented hosted route for this exact model is Google’s Gemini API unless another provider explicitly lists the same model. CometAPI may still be relevant to a broader application that uses other hosted Gemini or non-Gemini models for fallback and task routing. That is a hybrid architecture decision, not a Gemma 4 availability claim. The distinction matters more than forcing a product mention into the wrong model page. Use TurboFieldfare when prompts must remain on-device, the workload is text-only, Apple Silicon is already available, and local throughput is acceptable. Use the official API when the team needs image input, fast evaluation, burst capacity, managed security, or less operational work. Evaluate a hybrid route when private text tasks belong locally but public or burst traffic needs hosted capacity. Keep the routes measurable and test them with the same prompts, output limits, context lengths, and acceptance rules. Record time to first token, total latency, prefill time, decode throughput, peak memory, SSD reads, queue time, context length, structured-output validity, tool-call validity, accepted-task rate, retries, human correction time, and total cost. The 2GB result is valuable because it expands where a large MoE model can run. It does not make the other deployment dimensions optional. Gemma 4 26B in 2GB Is Real. The Headline Is Still Misleading. https://pub.towardsai.net/gemma-4-26b-in-2gb-is-real-the-headline-is-still-misleading-77ff61012b1b was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.