A 4 GB Laptop GPU vs a 6-Core CPU on Gemma 4, Re-Measured in ABBA Order: 4.1x

A developer benchmarked a 4 GB GeForce GTX 1650 Ti Max-Q against an Intel Core i7-10750H in the same Lenovo Yoga 9 chassis for serving the Gemma 4 E2B QAT Q4_0 GGUF via llama.cpp, finding the GPU decodes 4.14x faster, prefills 3.42x faster and completes requests 3.62x faster end to end. Running the passes in ABBA order to cancel thermal drift moved the decode ratio only from 4.09x to 4.17x, indicating a single-order benchmark on this laptop is off by roughly 2%. Both builds came from one llama.cpp commit (f95b0d9, build 318) on Debian sid with gcc 16.2.0 and CUDA 13.4, differing by a single flag, and the project also ships Python MCP tools for managing the llama.cpp deployment.

This article provides a step by step guide to measuring a laptop GPU against the CPU in the same chassis for serving Gemma 4 E2B through llama.cpp, with both builds on a fresh Debian sid toolchain and the passes run in ABBA order so heat cannot pick the winner. A suite of Python MCP tools is built to simplify management of the llama.cpp hosted deployment. https://github.com/xbill9/gemma4-dev https://github.com/xbill9/gemma4-dev The GTX 1650 Ti decodes 4.14x faster than the i7-10750H it shares a chassis with, prefills 3.42x faster and finishes requests 3.62x faster end to end. Running the two devices in one order and then the other moves that decode ratio from 4.09x to 4.17x, so a single-order benchmark on this laptop is off by about 2%. The same 3.35 GB quantization-aware GGUF, google/gemma-4-E2B-it-qat-q4 0-gguf , served by llama-server twice on one laptop: once on the CPU, once on the GPU. Both builds come from one llama.cpp commit and the two command lines differ by one flag. An earlier run of this comparison is published as A 4 GB Laptop GPU Beats a 12-Core CPU by 4.3x on Gemma 4 https://dev.to/gde/a-4-gb-laptop-gpu-beats-a-12-core-cpu-by-43x-on-gemma-4-4150 . This one re-measures it after the laptop moved to Debian sid, with three changes: a newer llama.cpp commit built on gcc 16.2 and CUDA 13.4, thread flags set for this CPU's real core count, and a run order that cancels thermal drift. nvcc 13.4 here cmake and a host compiler CUDA accepts — gcc 16.2.0 here python3 , no virtualenv local-llamacpp-1650ti-2b-q4 0/ and local-llamacpp-cpu-2b-q4 0/ side by side | Machine | Lenovo Yoga 9 15IMH5 | | CPU | Intel Core i7-10750H, 6 cores / 12 threads, AVX2 | | GPU | GeForce GTX 1650 Ti with Max-Q Design, 4096 MiB, compute capability 7.5 | | OS | Debian GNU/Linux forky/sid, kernel 7.2.6 | | Compiler | gcc 16.2.0 | | CUDA | 13.4 V13.4.92 | | llama.cpp | f95b0d9 , build 318 | The CPU and the Max-Q card sit under one cooling system, which is why the run order below matters. A distribution upgrade can move the compiler past what CUDA accepts, and a CUDA upgrade can drop old GPUs. Both were close here. bash $ nvidia-smi --query-gpu=name,compute cap,memory.total,driver version --format=csv name, compute cap, memory.total MiB , driver version NVIDIA GeForce GTX 1650 Ti with Max-Q Design, 7.5, 4096 MiB, 615.71.09 $ nvcc --list-gpu-arch compute 75 compute 80 compute 86 ... The card is compute capability 7.5, and compute 75 is the first entry CUDA 13.4 lists. This laptop's GPU is the oldest architecture the current toolkit still builds for. bash $ gcc --version gcc Debian 16.2.0-3 16.2.0 $ grep -n 'later than 16' /usr/local/cuda/include/crt/host config.h 137: error -- unsupported GNU version gcc versions later than 16 are not supported ... CUDA 13.4 accepts gcc up to 16, and sid ships 16.2. Each rig's Makefile carries its own build target. The GPU build targets only this card's architecture, sm 75; the CPU build turns every GPU backend off. local-llamacpp-1650ti-2b-q4 0 cmake -S ~/llama.cpp -B ~/llama.cpp/build -DGGML CUDA=ON -DCMAKE CUDA ARCHITECTURES=75 -DGGML NATIVE=ON -DCMAKE BUILD TYPE=Release cmake --build ~/llama.cpp/build --config Release -j6 --target llama-server llama-bench local-llamacpp-cpu-2b-q4 0 cmake -S ~/llama.cpp -B ~/llama.cpp/build-cpu -DGGML CUDA=OFF -DGGML VULKAN=OFF -DGGML NATIVE=ON -DCMAKE BUILD TYPE=Release cmake --build ~/llama.cpp/build-cpu --config Release -j6 --target llama-server llama-bench Both binaries report the same commit and compiler: $ ~/llama.cpp/build/bin/llama-server --version version: 0.4.1-dev build 318, commit f95b0d9 built with GNU 16.2.0 for Linux x86 64 $ ~/llama.cpp/build-cpu/bin/llama-server --version version: 0.4.1-dev build 318, commit f95b0d9 built with GNU 16.2.0 for Linux x86 64 And only one of them can see the card: $ ~/llama.cpp/build/bin/llama-server --list-devices Available devices: CUDA0: NVIDIA GeForce GTX 1650 Ti with Max-Q Design 3732 MiB, 3671 MiB free $ CUDA VISIBLE DEVICES= ~/llama.cpp/build-cpu/bin/llama-server --list-devices Available devices: none cmake --build -j with no number starts every CUDA source file at once. llama.cpp has about 188 of them, each nvcc forks cicc and cudafe++ , and a 15 GiB laptop runs out of memory. At -j6 the full CUDA build peaked at 4.0 GiB and never touched swap, so the limit costs nothing. Both Makefiles pass -j$ BUILD JOBS with a default of 6. After a compiler or CUDA upgrade, delete the build directory before configuring. A stale CMakeCache.txt keeps the old compiler, and /usr/local/cuda is a symlink, so the cached path looks current. Both devices serve on 127.0.0.1:8080 , so the endpoint, the prompts and the benchmark script stay fixed while the device changes underneath. The flags match except for -ngl , the number of layers offloaded to the GPU: llama-server -m gemma-4-E2B q4 0-it.gguf --host 127.0.0.1 --port 8080 \ -ngl {0|99} -c 8192 -ctk f16 -ctv f16 -fa 1 -t 6 -tb 12 --parallel 1 --metrics -t 6 -tb 12 matches the CPU's six physical cores and twelve threads, and both devices get it. GPU decode barely responds to thread count, so matching them costs the GPU nothing and removes one difference between the two runs. For the GPU side there is a small wrapper, llamacpp-1650ti , that reads these values out of the rig's tpu.env , starts the server in the background and checks which device answered: bash $ llamacpp-1650ti start llamacpp-1650ti: starting /home/xbill/models/gemma-4-E2B-it-qat-q4 0/gemma-4-E2B q4 0-it.gguf on 127.0.0.1:8080 -ngl 99 llamacpp-1650ti: pid 88166, log /home/xbill/gemma4-dev/local-llamacpp-1650ti-2b-q4 0/run/llama-server.log llamacpp-1650ti: waiting up to 180s for http://127.0.0.1:8080/health 0s VRAM 3 MiB, 0 % llamacpp-1650ti: healthy after 2s -- http://127.0.0.1:8080 llamacpp-1650ti: device=gpu · pid=88166 · -ngl 99 · mapped: ggml-cuda, libcublas, libcuda, libcudart · /home/xbill/llama.cpp/build/bin/llama-server $ llamacpp-1650ti stop llamacpp-1650ti: sent SIGTERM to pid 88166; VRAM is released on exit The wrapper refuses to start if anything already holds the port, and refuses to stop the CPU build if that is what holds it. Both devices answer the same URL with the same JSON, so the HTTP response cannot say which one produced it. The rigs' attest.py reads it from /proc instead: /proc/<pid /exe for the binary, /proc/<pid /maps for the GPU libraries it has loaded, /proc/<pid /cmdline for the real -ngl . The benchmark script refuses to start if the answer is the wrong device, and writes it at the top of every log: arm: device=cpu · pid=72478 · -ngl 0 · /home/xbill/llama.cpp/build-cpu/bin/llama-server arm: device=gpu · pid=74308 · -ngl 99 · mapped: ggml-cuda, libcublas, libcuda, libcudart · /home/xbill/llama.cpp/build/bin/llama-server maps is used because llama.cpp loads its CUDA backend at runtime, where ldd does not see it. Each report also records the SHA-256 of the executable that ran: 9c88c7821fcc11a6… for the CPU build and 9f2a8b0b6ed5365d… for the GPU build, the same on both passes of each. The MCP servers expose the same check as a tool, so an agent that starts a server can confirm what it started before it measures anything. Each device gets two passes, in the order CPU, GPU, GPU, CPU. Each device then has one early pass and one late pass, so a slow drift in the laptop's temperature over the session affects both equally. Before every pass the script waits at least 120 seconds and until the CPU package is at or below 50 °C and the GPU at or below 45 °C. 13:20:12 pkg=38C gpu=36C throttle=35105 START cpu pass1 13:27:57 pkg=85C gpu=52C throttle=44709 END cpu pass1 rc=0 13:30:00 pkg=45C gpu=40C throttle=44730 START gpu pass1 13:32:12 pkg=79C gpu=56C throttle=45150 END gpu pass1 rc=0 13:34:15 pkg=42C gpu=39C throttle=45150 START gpu pass2 13:36:26 pkg=78C gpu=55C throttle=48034 END gpu pass2 rc=0 13:38:29 pkg=41C gpu=39C throttle=48060 START cpu pass2 13:46:31 pkg=82C gpu=53C throttle=67346 END cpu pass2 rc=0 throttle is the CPU package's cumulative thermal-throttle counter. CPU pass 1 added 9,604 events, the two GPU passes 420 and 2,879, and CPU pass 2 added 19,279. A CPU pass takes about eight minutes and a GPU pass about two. Each pass is the same sweep: four prompt lengths by two output lengths, three repeats per cell, one request at a time. python3 sweep.py --base http://127.0.0.1:8080/v1 --out <run /passN --rig <rig --expect-device <cpu|gpu Each figure is the mean of the two passes' per-cell medians. Decode is the token rate measured off the response stream; TTFT is time to first token. | in tok | out tok | CPU decode | GPU decode | GPU lead | CPU TTFT ms | GPU TTFT ms | GPU lead | |---|---|---|---|---|---|---|---| | 94 | 32 | 18.31 | 71.89 | 3.93x | 1084 | 381 | 2.84x | | 94 | 128 | 17.96 | 71.57 | 3.98x | 1171 | 385 | 3.04x | | 516 | 32 | 17.06 | 70.61 | 4.14x | 5528 | 1608 | 3.44x | | 516 | 128 | 17.19 | 70.35 | 4.09x | 5527 | 1610 | 3.43x | | 998 | 32 | 16.10 | 69.25 | 4.30x | 11071 | 3242 | 3.41x | | 998 | 128 | 16.77 | 69.38 | 4.14x | 11057 | 3246 | 3.41x | | 1959 | 32 | 16.14 | 68.52 | 4.25x | 23118 | 6497 | 3.56x | | 1959 | 128 | 16.37 | 68.22 | 4.17x | 22718 | 6496 | 3.50x | Medians over the eight cells: decode 4.14x , prefill 3.42x , end to end 3.62x . Decode stays nearly flat on both devices across a 21x range of prompt length 94 to 1959 tokens , from 18.31 down to 16.14 tok/s on the CPU and from 71.89 down to 68.22 on the GPU, while TTFT grows in proportion to the prompt on both. Decode reads the model once per token and is limited by memory bandwidth; prefill does arithmetic over the whole prompt. The GPU wins both, by different margins, and the prefill margin grows with the prompt from 2.84x to 3.56x. By an order of magnitude. The decode ratio stays between 3.93x and 4.30x in every cell. The noise is the drift between a device's two passes: | | decode drift, pass 1 to pass 2 | worst cell | |---|---|---| | GPU | median +0.6% , range 0.0% to +1.3% | — | | CPU | median -1.1% , range -9.3% to +0.1% | 998 in / 32 out: 16.89 → 15.32 | Within a pass, the three repeats of a cell spread by at most 1.14% on the GPU. On the CPU they spread by at most 2.69% in pass 1 and 14.05% in pass 2, which had twice the throttle events. CPU time to first token also rose in pass 2, by up to 7.4% at the longest prompt, while the GPU's moved by at most 0.9%. All of the measurable noise in this comparison comes from the CPU, and it tracks heat. Each ordering can be read on its own. CPU pass 1 against GPU pass 1 is the CPU-first design; GPU pass 2 against CPU pass 2 is GPU-first. | order | decode ratio | prefill ratio | |---|---|---| | CPU first, then GPU | 4.09x | 3.38x | | GPU first, then CPU | 4.17x | 3.47x | | ABBA, both | 4.14x | 3.42x | Running the CPU first understates the GPU's lead, because the CPU pass runs while the laptop is still cool and the GPU pass inherits its heat. On this laptop the effect is about 2% of the ratio. ABBA order removes it for the cost of one extra pass per device. llama.cpp can put more than one token in a streamed chunk, so a decode rate computed from chunk timing undercounts. Recomputed as completion tokens over the measured decode time, the rates read 16.77-20.27 tok/s on the CPU and 69.87-79.58 on the GPU, and the median ratio stays at 4.14x . Both devices run the same server code, so the undercount cancels in the ratio. It does not cancel when comparing either absolute figure against a different server. llama-server reuses the key/value cache for a prompt prefix it has seen, which makes time to first token look faster than prefill really is. Each prompt starts with a unique prefix, and --metrics exposes a counter that confirms it: cpu pass1: llamacpp:prompt tokens total 28553 llamacpp:prompt tokens cached total 0 cpu pass2: llamacpp:prompt tokens total 28553 llamacpp:prompt tokens cached total 0 gpu pass1: llamacpp:prompt tokens total 28553 llamacpp:prompt tokens cached total 0 gpu pass2: llamacpp:prompt tokens total 28553 llamacpp:prompt tokens cached total 0 None of the 28,553 prompt tokens in any pass came from the cache. | | 🥇 GTX 1650 Ti | 🥈 i7-10750H | |---|---|---| | Decode, single stream | 68.22-71.89 tok/s | 16.10-18.31 tok/s | | Time to first token, 1959-token prompt | 6.5 s | 22.7-23.1 s | | Pass-to-pass drift | ≤1.3% | up to 9.3% | | Throttle events per pass | 420 and 2,879 | 9,604 and 19,279 | | Pass length | about 2 minutes | about 8 minutes | The GPU, for any interactive use of this model on this laptop. It is about four times faster on every prompt length measured, it is steadier, and it heats the shared package far less than the CPU does for the same work. The CPU build is still worth keeping: it serves the same model at 16-18 tok/s when the card is busy or absent. It is also the device to benchmark with the most care, because its numbers move with the laptop's temperature. The earlier run measured 4.27x decode and 3.63x prefill. Three things changed at once — the llama.cpp commit c6824a9 to f95b0d9 , the thread flags -t 4 -tb 8 to -t 6 -tb 12 and the run order — so the difference between the two results cannot be assigned to any one of them. The more-threads explanation fits the direction, since CPU time to first token fell and the prefill ratio narrowed with it, but this run does not isolate it. The goal of this article was to re-measure a 4 GB laptop GPU against the 6-core CPU in the same chassis for serving Gemma 4 E2B, after a toolchain upgrade and with thermal drift controlled. The key to the solution was ABBA ordering with a temperature gate, identical flags on both devices apart from -ngl , and reading the serving device from /proc on every pass. The results were: Scope: one laptop, one GGUF, llama.cpp f95b0d9 built twice with gcc 16.2 and CUDA 13.4, eight paired cells, two passes per device in ABBA order, three repeats per cell per pass, one request at a time, prompt cache verified cold, page cache warm throughout. The strategy for using MCP for local accelerator comparison was validated with an incremental step by step approach.