{"slug": "28-9m-parameter-llm-runs-locally-on-esp32-s3-at-9-tokens-s", "title": "28.9M-parameter LLM runs locally on ESP32-S3 at 9 tokens/s", "summary": "Slava S. (slvDev) has optimized a 28.9M-parameter LLM to run locally on an ESP32-S3 development board at about 9 tokens/s, generating short stories on an I2C display. The project, called 'esp32-ai', requires an ESP32-S3 with at least 512KB SRAM+8MB SPRAM and 16MB flash to store the 14.9MB 4-bit model file, and was tested on an ~$8 board. This differs from previous ESP32-based LLM projects that used the board as a gateway to online LLMs.", "body_md": "Slava S. (slvDev) has optimized a 28.9M-parameter LLM running locally on an ESP32-S3 development board at around 9 tokens/s while generating text, or more exactly, telling short stories. We’ve written about LLMs and ESP32 in the past, but usually the boards are used as low-power gateways to interact with online LLMs hosted on powerful servers in data centers. Such products or projects include HiWonderLLM “smart chat module”, the ESP-Claw framework, or the ESP32 Agent Dev Kit. Slava’s “esp32-ai” project is quite different, as everything runs on the ESP32-S3 microcontroller. This won’t work on all ESP32-S3 as you’ll need enough memory and storage to store the 14.9MB (4-bit) model file, and Slava tested his demo on an ~$8 ESP32-S3 board with 512KB SRAM+8MB SPRAM and 16MB flash connected to a small I2C display to display short stories. Dave Bennett already did something similar with the ESP32-LLM project, but it only relied [...]\n\nThe post [28.9M-parameter LLM runs locally on ESP32-S3 at 9 tokens/s](https://www.cnx-software.com/2026/08/03/28-9m-parameter-llm-runs-locally-on-esp32-s3-at-9-tokens-s/) appeared first on [CNX Software - Embedded Systems News](https://www.cnx-software.com).", "url": "https://wpnews.pro/news/28-9m-parameter-llm-runs-locally-on-esp32-s3-at-9-tokens-s", "canonical_source": "https://www.cnx-software.com/2026/08/03/28-9m-parameter-llm-runs-locally-on-esp32-s3-at-9-tokens-s/", "published_at": "2026-08-03 10:40:32+00:00", "updated_at": "2026-08-05 11:52:24.147183+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure"], "entities": ["Slava S.", "ESP32-S3", "esp32-ai", "CNX Software", "Dave Bennett", "ESP32-LLM", "HiWonderLLM", "ESP-Claw"], "alternates": {"html": "https://wpnews.pro/news/28-9m-parameter-llm-runs-locally-on-esp32-s3-at-9-tokens-s", "markdown": "https://wpnews.pro/news/28-9m-parameter-llm-runs-locally-on-esp32-s3-at-9-tokens-s.md", "text": "https://wpnews.pro/news/28-9m-parameter-llm-runs-locally-on-esp32-s3-at-9-tokens-s.txt", "jsonld": "https://wpnews.pro/news/28-9m-parameter-llm-runs-locally-on-esp32-s3-at-9-tokens-s.jsonld"}}