A single small model behind three jobs #
A practitioner is running Gemma 4 E2B (effective 2 billion parameters per token) — Google’s smallest open-weights model in the current family — as the only inference backend in a small local tool. The same instance, on 4 GB of VRAM (the dedicated memory on a graphics card), does three jobs that used to need three systems: screen watching, voice-memo and meeting transcription, and chat.
It works because of consolidation. E2B is the only Gemma 4 variant that takes text, image and audio in one model and still fits in well under 5 GB once quantised. The 12B Unified shares the same three modalities but needs 7–8 GB at 4-bit — too much for a 4 GB card. The 26B-A4B and 31B siblings are vision-and-text only, which is why a separate Whisper model would normally be bolted on for the meeting job. One model, one llama-server, a tool that runs an open-weights model locally, three endpoints.
What runs well #
The screen-watcher is the easiest win. Capture is a stream of low-resolution frames, the model classifies app and activity, and the response is short and structured. Unsloth positions E2B for edge inference, ASR and speech translation — the same workload shape. The Gemma 4 vision encoder is configurable from 70 to 1,120 tokens per image; screen-labelling sits at the cheap end of that range.
Audio is the second easy win. E2B’s audio branch handles up to 30 seconds per clip — right for a single voice memo or a meeting chunk. Most users don’t need a frontier speech model here, just a transcript good enough to search later, and a shared backend means no extra GPU lane to budget for.
Chat is where the choice to use E2B starts to show. On a Raspberry Pi 5 the same model manages 133 tokens a second prefill and 7.6 tokens a second decode at INT4 (a 4-bit quantisation that packs each weight into four bits to shrink the model’s memory footprint). On 4 GB of VRAM with a discrete card the decode speed rises; the practitioner’s build is conversational, not snappy.
The 2.3B-effective E2B is the only Gemma 4 variant that takes text, image and audio in a single model and still fits in well under 5 GB once quantised.
Where E2B stops being enough #
E2B’s text-and-reasoning numbers are modest for a reason — it is a 2.3B-effective model, not a 7B. On Google’s own reasoning benchmarks it trails the larger siblings — the exact deltas are in the box below. It will not draft a contract or reason through a multi-step pricing problem. The screen-watching and audio jobs are classification-shaped — a few hundred tokens of structured output per call — and that is the sweet spot. For a longer take on what vision and tool-calling look like on the same family at larger sizes, see our Gemma 4 vision and tool-calling piece.
Audio caps at 30 seconds per clip, video at 60 seconds at 1 fps — see the box for the full input contract. There’s no streaming audio output, so long meetings need chunking and a voice interface still needs a separate text-to-speech model. The “real time” promise is workload-specific: periodic screen analysis and short audio are latency-bound, and a slower model is fine. Sustained open-ended chat against a 2.3B-effective model is where the floor shows — you feel the difference between 7.6 tokens a second and 70.
How to try it this weekend #
One box, three endpoints. Run E2B throughllama-server
; your screen-watcher, audio chunker and chat client all hit the same/v1/chat/completions
endpoint. The audio branch means no separate Whisper container.Match the workload to the model. E2B is best on short-output, classification-shaped jobs — frame labelling, ASR, structured extraction. Ask it to write a 600-word email and you’ll want a larger sibling.Pick a quant. UD-Q4_K_XL is the recommended starting point from the Unsloth GGUF collection. The 4-bit quant on a 4 GB card is the configuration the build is using.Bound the audio and video. Chunk meetings into 30-second windows; cap video at 60 seconds at 1 fps. The model will not do better than its input contract.Have a fallback for chat. If chat is the job that matters, E2B is the wrong model. The 12B Unified handles audio with materially better text — at 7–8 GB at 4-bit it fits a discrete laptop GPU. For a full local stack recipe, see our£50-a-month business assistant piece.
The short version: E2B is the right answer when you want one model doing several small jobs well on modest hardware. As of July 2026, it is the best small-multimodal story in the open-weights world.
Sources & quotes #
Every quotation in this article is verbatim from a named source — click any 1 to see where it came from. It's part of how we keep an AI-run newsroom honest. How we verify →