POCKET-27B-Pi: Running a 27B Coding Agent on a Single 24GB GPU with a 16.5GB GGUF File VIDRAFT released POCKET-27B-Pi, a Qwen3.8-27B coding agent with the pi adapter merged directly into the model weights and shipped as a single 16.5GB Q4_K_M GGUF file that fits on one 24GB GPU with a full 64K context window. On Terminal-Bench 2.1's 48 shared tasks it completed 28 (58.3%) versus 19 (39.6%) for the base model, a 47% relative improvement, and the Q4_K_M file matched the larger Q6_K variant on 62 overlapping tasks. The model is published under Apache-2.0 on Hugging Face and served locally through llama.cpp's llama-server with an OpenAI-compatible API. TL;DR: VIDRAFT's POCKET-27B-Pi merges a Qwen3-based 27B coding agent adapter directly into model weights and ships it as a single Q4 K M GGUF file weighing just 16.5GB — small enough to fit on a single 24GB GPU with a full 64K context window. On Terminal-Bench 2.1's 48 shared tasks, it completes 28 58.3% , versus 19 39.6% for the base Qwen3.8-27B — a 47% relative improvement — at 3.3× smaller file size than the full-precision pi build. POCKET-27B-Pi is a local-deployment coding agent released by VIDRAFT. It is built on a Qwen3.8-27B backbone and ships with the pi coding agent adapter already merged into the model weights — no separate adapter loading required at inference time. Two quantization variants are published to the Hugging Face repository FINAL-Bench/POCKET-27B-Pi-GGUF under an Apache-2.0 license : The key engineering proposition here isn't just the parameter count. It's a verifiable memory budget : this model, with a 64K context window open, fits inside a single 24GB GPU. That targets workstations, small-form-factor desktops, and on-premise developer machines — deployments that cannot or should not depend on a cloud API. The memory efficiency comes from the base model's hybrid attention architecture . According to the model's public config.json : This distinction matters enormously for KV cache sizing. Only the 16 full-attention layers accumulate per-token K and V tensors. With num key value heads = 4 and head dim = 256 , the per-token KV cost in f16 works out to a fixed, calculable figure. At 64K context, the f16 KV cache is approximately 4.0 GiB . llama.cpp's quantized cache options bring this down further: q8 0 cache: ~2.13 GiB q4 0 cache: ~1.13 GiB Combining the Q4 K M weights ~15.4 GiB with a 64K f16 KV cache lands at roughly 19.55 GiB before compute buffers — leaving headroom inside a 24GB card. Switching to q8 0 cache reclaims about 1.9 GiB, which matters if you're running concurrent agent sessions. There's also a decode-speed angle. On consumer hardware, tokens-per-second is largely a memory bandwidth problem : throughput ≈ bandwidth ÷ bytes-read-per-token. A file that's ~3.3× smaller than the full-precision build has a proportionally higher theoretical token throughput ceiling on the same GPU. For a time-budgeted coding agent each task gets one hour , faster decoding directly translates to more reasoning turns completed before the clock runs out. All numbers below are from VIDRAFT's own Terminal-Bench 2.1 evaluation. Thinking was disabled; each task received one attempt and a one-hour wall-clock budget; eight agent sessions ran in parallel on a single GPU. On the 48 tasks completed by all evaluated builds: | Model | Tasks Completed | Completion Rate | |---|---|---| | POCKET-27B-Pi Q4 K M | 28 | 58.3% | | Base Qwen3.8-27B | 19 | 39.6% | On 50 shared tasks direct comparison : Q4 K M vs. Q6 K on 62 overlapping tasks: Both completed 39 tasks — the smaller file matched the larger one exactly, supporting Q4 K M as the recommended delivery format. Agent efficiency 50-task set : The turn-count delta is significant: the pi agent reaches solutions in roughly half the turns, freeing up time budget for more tasks within the hour window. The model files are publicly available on Hugging Face. You can download and serve locally using llama.cpp's llama-server : Download the Q4 K M variant huggingface-cli download FINAL-Bench/POCKET-27B-Pi-GGUF \ --include " .Q4 K M.gguf" --local-dir ./pocket-27b-pi Serve with 64K context, thinking disabled, quantized KV cache for headroom llama-server \ -m ./pocket-27b-pi/