Test a custom prompt against an already-trained, already-flashed ESP32-S3 model

A developer released a shell script that lets users test custom prompts against an already-trained ESP32-S3 model without retraining or rewriting the 15MB model partition. The script tokenizes the prompt, patches the firmware sketch, and recompiles and flashes only the app, leaving the model untouched.

| /usr/bin/env bash | | | esp32-ai: test a custom prompt against an already-trained, already-flashed model. | | | Does NOT retrain and does NOT rewrite the 15MB model partition. | | | | | | Usage: | | | ./run custom prompt.sh "Once there was a robot" | | | set -euo pipefail | | | if "$ " -lt 1 ; then | | | echo "Usage: $0 \"Your custom prompt\"" | | | exit 1 | | | fi | | | PROMPT="$1" | | | REPO DIR="esp32-ai" | | | if -f "$REPO DIR/pyproject.toml" ; then | | | cd "$REPO DIR" | | | elif -f "pyproject.toml" ; then | | | echo "Can't find the esp32-ai repo. Run this from the same folder as build and flash.sh, after it has cloned esp32-ai." | | | exit 1 | | | fi | | | if -f "data/bpe32768.json" ; then | | | echo "data/bpe32768.json not found. Run build and flash.sh at least once first it trains the tokenizer ." | | | exit 1 | | | fi | | | ---- Detect port ---- | | | CANDIDATE PORTS= /dev/cu.usbmodem | | | if -e "${CANDIDATE PORTS 0 }" ; then | | | echo "No /dev/cu.usbmodem device found. Is the board plugged in?" | | | exit 1 | | | elif "${ CANDIDATE PORTS @ }" -eq 1 ; then | | | ESP32 PORT="${CANDIDATE PORTS 0 }" | | | else | | | select choice in "${CANDIDATE PORTS @ }"; do | | | -n "$choice" && ESP32 PORT="$choice" && break | | | done | | | fi | | | echo " Using port: $ESP32 PORT" | | | ---- Tokenize the custom prompt with the same tokenizer gen assets.py uses ---- | | | echo " Tokenizing prompt: \"$PROMPT\"" | | | TOKENIZE SCRIPT="$ mktemp /tmp/tokenize prompt XXXX.py " | | | cat "$TOKENIZE SCRIPT" << 'PYEOF' | | | import sys | | | from tokenizers import Tokenizer | | | tok = Tokenizer.from file "data/bpe32768.json" | | | prompt = sys.argv 1 | | | ids = tok.encode prompt .ids | | | print f'prompt "{prompt}" - {len ids } ids: {ids}' | | | print f"round-trip: {tok.decode ids r}" | | | print "static const int PROMPT IDS = {" + ", ".join map str, ids + "};" | | | PYEOF | | | GEN OUTPUT="$ uv run python "$TOKENIZE SCRIPT" "$PROMPT" " | | | rm -f "$TOKENIZE SCRIPT" | | | echo "$GEN OUTPUT" | | | PROMPT IDS LINE="$ echo "$GEN OUTPUT" | grep -o 'static const int PROMPT IDS\ \ = { ^} };' " | | | if -z "$PROMPT IDS LINE" ; then | | | echo " Couldn't parse a PROMPT IDS line out of the tokenizer output. Aborting." | | | exit 1 | | | fi | | | echo " New prompt line: $PROMPT IDS LINE" | | | ---- Paste it into the sketch ---- | | | SKETCH FILE="firmware/esp32 llm/esp32 llm.ino" | | | sed -i.bak -E "s/static const int PROMPT IDS\ \ = \{ ^} \};/${PROMPT IDS LINE//\//\\/}/" "$SKETCH FILE" | | | ---- Rebuild and reflash just the app model partition untouched ---- | | | FQBN='esp32:esp32:esp32s3:UploadSpeed=921600,USBMode=hwcdc,CDCOnBoot=default,UploadMode=default,CPUFreq=240,FlashMode=qio,FlashSize=16M,PartitionScheme=custom,PSRAM=opi,DebugLevel=info' | | | echo " Recompiling firmware with new prompt..." | | | arduino-cli compile \ | | | --fqbn "$FQBN" \ | | | --build-property compiler.optimization flags=-O3 \ | | | --build-path /tmp/esp32-llm-uart \ | | | firmware/esp32 llm | | | echo " Flashing app model partition is untouched ..." | | | arduino-cli upload -p "$ESP32 PORT" \ | | | --fqbn "$FQBN" \ | | | --input-dir /tmp/esp32-llm-uart firmware/esp32 llm | | | echo " Opening serial monitor. Press the board's RESET button now." | | | arduino-cli monitor -p "$ESP32 PORT" --config baudrate=115200 |