Mastering Edge AI on Raspberry Pi with LiteRT and Gemma Google AI Edge's LiteRT and LiteRT-LM enable deployment of Gemma 4 E2B on Raspberry Pi 5, achieving 99 tokens/sec prefill and 9 tokens/sec decode with a peak memory footprint of 1432 MB, powering the Reachy Mini robot for fully local, real-time perception and reaction. The integration, including GPU inference via LiteRT's WebGPU (Vulkan) backend through ML Drift, supports autonomous agents, smart cameras, and social robotics with zero cloud dependencies. Imagine building a fully autonomous robot that can see, hear, and react to its environment in real time, completely offline on a single compact device like Raspberry Pi https://www.raspberrypi.com/ . Edge AI unlocks this exact autonomy. It enables developers to build highly secure and self-contained systems like intelligent robotics and local AI agents with zero cloud dependencies, ultra-low latency, and total data privacy. We’ve made running edge AI on the Raspberry Pi a breeze with Google AI Edge’s LiteRT https://developers.googleblog.com/litert-the-universal-framework-for-on-device-ai/ , high-performance production-proven on-device inference runtime. LiteRT allows you to seamlessly deploy everything from classical ML models to state-of-the-art LLMs right out of the box across platforms. By delivering optimized execution and hyper-efficient memory usage on both CPU and GPU, LiteRT maximizes your Raspberry Pi's full computing potential. This hardware-software synergy shines brightest when paired with Gemma https://deepmind.google/models/gemma/ , Google’s family of lightweight open models. To show you what’s possible, we'll demonstrate how Gemma and LiteRT on a Raspberry Pi 5 can power the Reachy Mini robot to perceive and react to its environment entirely locally in real time. Read on to get started with your own deployment. Gemma models are well suited for building autonomous agents, smart cameras, and social robotics that can reason and execute complex, multi-step workflows directly on your Raspberry Pi. To accommodate different hardware constraints, the Gemma family of models provides several highly efficient options: Through LiteRT-LM https://developers.google.com/edge/litert-lm , a specialized orchestration layer on top of LiteRT, developers can seamlessly deploy Gemma right out of the box. Under the hood, sophisticated CPU acceleration via LiteRT and XNNPACK https://github.com/google/xnnpack ensures the Gemma family of models is highly optimized for resource efficiency and low-latency execution directly on the Raspberry Pi. On a Raspberry Pi 5, LiteRT-LM delivers a robust performance for Gemma 4 E2B, achieving 99 tokens/sec for prefill and 9 tokens/sec for decode, all while maintaining a remarkably low peak memory footprint of just 1432 MB. This brings Gemma’s highly responsive, general-purpose intelligence to Raspberry Pi . Thanks to Gemma 4 E2B's highly efficient tokenizer, which packs more text into fewer tokens averaging ~4.2 characters per token , LiteRT-LM achieves an impressive end-to-end generation speed of ~27.3 characters per sec, roughly 300 words per minute wpm , in the Reachy Mini voice demo. This throughput makes Gemma 4 E2B excellent for real-time speech and translation tasks , delivering text at twice the speed of normal human speech ~150 wpm . Explore more ready-to-use open models to run on Raspberry Pi from the LiteRT Hugging Face Community https://huggingface.co/litert-community . On the Raspberry Pi 5, the quad-core ARM Cortex-A76 CPU is a raw computing powerhouse, delivering ~153.6 GFLOPS FP32 and up to ~2.0 TOPS INT8 . In comparison, the integrated Broadcom VideoCore VII GPU is clocked at 800 MHz and offers a peak of ~76.8 GFLOPS FP32 and ~0.24 TOPS INT8 . While the CPU possesses a massive capacity advantage, the GPU introduces heterogeneous parallel execution , a paradigm critical for real-time edge applications. Rather than saturating the CPU, developers can delegate tasks across both processors to optimize overall system and thermal efficiency. For example, by offloading continuous vision or audio models to the VideoCore VII GPU, it preserves high-priority CPU cycles for overall system monitoring, pipeline orchestration, or computationally demanding LLM inference. As such, we have enabled GPU inference on the Raspberry Pi with LiteRT’s WebGPU Vulkan backend via ML Drift . This integration allows you to run a wide range of computer vision, audio, and embedding models directly from the LiteRT Hugging Face Community https://huggingface.co/litert-community , including seamless support for popular MediaPipe models https://developers.google.com/edge/mediapipe/solutions/guide available solutions , Ultralytics YOLO models https://docs.ultralytics.com/integrations/litert , Moonshine https://huggingface.co/litert-community/moonshine-tiny/tree/main , and much more. The table below demonstrates the CPU and GPU latency of running classic computer vision and audio models via LiteRT: The Reachy Mini pipeline is a powerful showcase of low-latency, real-time edge AI inference running entirely on the Raspberry Pi 5. By leveraging LiteRT, the system splits intensive vision and language workloads into a concurrent, dual-processing architecture across CPU and GPU. Here is how the parallel architecture works under the hood to ensure seamless interactions: See the full source code of the Reachy demo in the LiteRT Samples Github repo https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert lm/reachy-voice-robot/ . LiteRT provides a comprehensive suite of tools that covers the full development cycle: conversion, quantization, benchmark, and inference. For a fast, frictionless setup, the most straightforward approach is using the LiteRT CLI https://github.com/google-ai-edge/LiteRT-CLI tool. Rather than requiring developers or coding agents to manually manage multiple independent libraries, the LiteRT CLI aggregates core edge workflows into a single, unified command set. You can now supercharge your development cycle by adding the LiteRT CLI skill https://github.com/google-ai-edge/LiteRT-CLI/blob/main/.agents/skills/litert cli/SKILL.md and other advanced LiteRT skills https://github.com/google-ai-edge/litert-samples/tree/main/skills into your AI coding agent, such as Google Antigravity https://antigravity.google/ . This empowers agents to autonomously orchestrate and execute complex, multi-stage machine learning workflows on your behalf. For example, you can easily build your own voice translator completely offline on a Raspberry Pi, like the Gemma Translator shown below. Link to Youtube Video https://www.youtube.com/watch?v=4dNry5zP0Jo visible only when JS is disabled Explore the complete implementation details in the Gemma Translator GitHub repo https://github.com/google-gemma/gemma-translator . For resource-constrained IoT devices, minimizing storage and memory overhead is critical. Without special optimization, generic AI runtimes often bundle heavy desktop or server dependencies. In contrast, LiteRT is engineered specifically for on-device deployment, maintaining an exceptionally lean and modular distribution. The table below compares the download footprint required to run LLM inference on a Raspberry Pi ARM64 Linux . You can install the LiteRT CLI and run your first model on a Raspberry Pi 5 with just a few simple commands. 1. Install LiteRT CLI To get started, install the LiteRT CLI via pip ideally within a virtual environment : pip install litert-cli 2. Run the model Download and run any compatible model directly from the LiteRT Hugging Face Community https://huggingface.co/litert-community . The code snippet below demonstrates how to execute Gemma 4 E2B e.g. gemma-4-E2B-it-litert-lm https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm on the Raspberry Pi 5. Run the model by providing your Hugging Face authentication token: export HUGGING FACE HUB TOKEN=