Over 100 AI agents and humans collaborated in a week-long challenge to make Google's open-weight model dramatically faster on a single GPU
Google DeepMind’s Gemma team and Hugging Face ran an experiment they called the “Fast Gemma Challenge,” and the results are worth paying attention to: a 5x improvement in inference speed for the Gemma 4 model, achieved by more than 100 AI agents and human participants working together over roughly six days.
The peak performance hit 491.8 tokens per second.
How the challenge actually worked #
The Fast Gemma Challenge ran from June 26 to July 2, with the specific target being the Gemma 4 E4B-it model. The constraint was deliberately tight: participants had to optimize inference using a single NVIDIA A10G GPU with just 24 GB of memory.
Participants submitted their optimizations through an OpenAI-compatible endpoint, and progress was tracked via a public leaderboard and shared dashboard.
The results, announced on July 10, split into two categories. The absolute peak performance reached 491.8 tokens per second. The best “lossless” submission, meaning optimizations that preserved the model’s output quality without any degradation, clocked in at 315 TPS. Both figures represent a roughly 5x jump from the baseline performance.
The techniques that got participants there included the vLLM inference engine, speculative decoding, quantization, IPC/ZMQ tuning, and integration with TGI (Text Generation Inference).
Why inference speed matters more than you think #
Making inference 5x faster on the same hardware means either serving 5x more users at the same cost or cutting your compute bill by 80% for the same workload.
The single-GPU constraint of the challenge was a deliberate choice. A10G cards are widely available through cloud providers like AWS at relatively modest cost. Proving that a capable model can run 5x faster on that kind of accessible hardware lowers the barrier to deploying competitive AI applications.
The bigger picture for AI markets #
Google and Hugging Face crowdsourced optimization work with AI agents participating as first-class contributors alongside human engineers, with more than 100 agents and humans working in parallel on a shared problem with a transparent leaderboard driving competitive pressure.
Investors watching the AI hardware space should note that if software optimization can deliver 5x speedups on existing hardware, that partially undermines the narrative that every AI workload requires the latest and most expensive chips. The A10G used in this challenge is an NVIDIA product, but the value proposition shifts when clever software makes mid-tier hardware punch above its weight.
Worth watching: whether Google integrates these community-sourced optimizations into its own Gemma deployment infrastructure, and whether Hugging Face uses challenges like this to drive adoption of its inference platform.
Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our