{"slug": "gemma-4-technical-report", "title": "Gemma 4 Technical Report", "summary": "Google DeepMind released Gemma 4, a new generation of open-weight multimodal language models featuring dense and Mixture-of-Experts architectures from 2.3B to 31B parameters. The models introduce a thinking mode for reasoning traces, improved vision and audio encoders, and a unified encoder-free architecture for the 12B variant, achieving state-of-the-art performance on STEM, multimodal, and long-context benchmarks.", "body_md": "arXiv:2607.02770v1 Announce Type: new\nAbstract: We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemma 4 model suite features dense and Mixture-of-Experts architectures, ranging from 2.3B to 31B parameters. Alongside improved vision and audio encoders for all model sizes, we propose a unified, encoder-free architecture for our 12B model, which ingests raw audio and image patches. Furthermore, we integrate a thinking mode, enabling Gemma models to generate reasoning traces prior to responding. We improve inference speed, memory, and compute efficiency, as well as long-context abilities through critical design choices. Gemma 4 establishes a leap in performance across STEM, multimodal, and long-context benchmarks, and rivals larger, frontier open models in human-rated tasks.", "url": "https://wpnews.pro/news/gemma-4-technical-report", "canonical_source": "https://arxiv.org/abs/2607.02770", "published_at": "2026-07-07 04:00:00+00:00", "updated_at": "2026-07-07 04:01:42.119679+00:00", "lang": "en", "topics": ["large-language-models", "generative-ai", "ai-research"], "entities": ["Google DeepMind", "Gemma 4", "Mixture-of-Experts"], "alternates": {"html": "https://wpnews.pro/news/gemma-4-technical-report", "markdown": "https://wpnews.pro/news/gemma-4-technical-report.md", "text": "https://wpnews.pro/news/gemma-4-technical-report.txt", "jsonld": "https://wpnews.pro/news/gemma-4-technical-report.jsonld"}}