How VIDRAFT Hit 510.58 TPS on Gemma-4: A Deep Dive into "The First Gemma Challenge" Win

VIDRAFT won 'The First Gemma Challenge' with a verified 510.58 tokens-per-second score on Google's Gemma-4-E4B-it model using a single NVIDIA A10G GPU, while a rival submission posted a faster raw number but failed the quality gate. The winning configuration, named vidraft-fw188-ctk49-n64-patchbridge-v1, leveraged software optimizations including a sliding window of 188, centroid top-k of 49, and a warm-up bridge to absorb compilation costs, achieving a perplexity of 2.3930 within the budget.

TL;DR:VIDRAFT topped "The First Gemma Challenge" leaderboard with a verified 510.58 tokens-per-second TPS score on google/gemma-4-E4B-it using a single NVIDIA A10G GPU — while a rival submission posted a faster raw number but failed the quality gate. This post breaks down the public configuration choices that made it possible, and what engineers can steal for their own inference tuning work. "The First Gemma Challenge" was a constrained inference-speed competition with two hard rules: one fixed GPU NVIDIA A10G , one fixed model google/gemma-4-E4B-it . Participants could not swap in stronger hardware or a lighter model. The only lever available was software-level optimization . The scoring metric was TPS Tokens Per Second , but with a simultaneous Perplexity PPL budget — submissions whose PPL exceeded approximately 2.42 were disqualified regardless of speed. The organizers also ran a blind re-evaluation against a held-out prompt set that participants never saw, which meant any configuration over-fitted to self-reported benchmarks would get caught. VIDRAFT's winning submission — configuration name vidraft-fw188-ctk49-n64-patchbridge-v1 — posted: A competing entry recorded 535.91 TPS, but its PPL landed at approximately 2.44, breaching the quality threshold. That's why the lower raw number was recognized as the verified SOTA. The public manifest.json for the winning configuration reveals three conceptual optimization pillars: SLIDING WINDOW=188 The KV-cache memory bandwidth is the dominant bottleneck during autoregressive generation. Restricting the attention window to only the most recent tokens reduces this pressure and increases throughput — but shrink the window too far and you lose context, causing PPL to spike. The value 188 is notably not a round number, which strongly suggests it was determined empirically rather than chosen from a default. The team overrode the model's text config.sliding window via HF OVERRIDES and enabled Flash Attention sliding FA SLIDING=1 to match. CENTROID TOP K=49 This parameter sits closer to the kernel level and affects both throughput and PPL simultaneously. According to the source analysis, values like 44, 48, and 49 were tested sequentially — the goal being to find the highest value that still kept PPL within budget. Bigger is not automatically better; it's a Pareto search under the quality constraint. The configuration uses: WARMUP BRIDGE=1 , WARMUP NUM PROMPTS=64 , WARMUP MAX TOKENS=1 , WARMUP SEED=42 . This fires 64 single-token dummy prompts before the timed benchmark begins, so that CUDA graph capture and JIT compilation costs are absorbed before the clock starts. The source article notes this warm-up was worth approximately 15 TPS — a significant margin in a competition decided by tens of TPS. Equally important: PRECACHE BENCH=0 is explicitly set, disabling a flag that would have inflated the self-reported TPS. The team chose to measure what the blind evaluator would actually see. SPECULATIVE CONFIG , with num speculative tokens=7 and method=mtp — a draft-then-verify approach that increases tokens generated per forward pass MAX MODEL LEN=4096 GPU MEMORY UTILIZATION=0.90 MAX NUM BATCHED TOKENS=512 MAX NUM SEQS=1 | Submission | TPS | PPL | Blind eval | |---|---|---|---| VIDRAFT vidraft-fw188-ctk49-n64-patchbridge-v1 | 510.58 | 2.3930 | ✅ Passed | | Competing entry | 535.91 | ~2.44 | ❌ Failed PPL 2.42 | The key takeaway: raw throughput rank and validated rank diverged because the quality gate was enforced on a held-out prompt distribution, not the participants' own test sets. The model used in the competition is publicly available from Google on Hugging Face: huggingface-cli download google/gemma-4-E4B-it The specific VIDRAFT configuration vidraft-fw188-ctk49-n64-patchbridge-v1 and any VIDRAFT-specific tooling are not confirmed as publicly released at the time of writing. Check VIDRAFT's Hugging Face organization https://huggingface.co/vidraft and their GitHub for updates. If access channels are announced, they will appear there first. Q: Why does the PPL threshold matter more than raw TPS in a "speed" competition? A: Because TPS without a quality floor is trivially gamed — you can degrade output until the model produces garbage very quickly. The PPL ceiling plus blind re-evaluation together enforce that the speed number reflects real, deployable inference quality. Q: Can I apply these same techniques to other models or GPUs? A: The concepts — quality-gated parameter search, warm-up separation, sliding window tuning, speculative decoding — are general inference engineering practices. The specific numeric values SLIDING WINDOW=188 , CENTROID TOP K=49 , etc. were tuned for google/gemma-4-E4B-it on a single A10G and should be treated as starting points, not copy-paste targets, for different hardware or model configurations. Q: What is speculative decoding method=mtp doing here? Originally reported by note 일본 2026-08-15 — source article.