Running an Autonomous AI Agent Stack on Android with Termux — No Cloud, Just a Phone

I’ve spent the last few months trying to turn an old Android phone into a self‑contained AI agent that can listen, reason, and act without ever reaching for a cloud service. The goal was simple: keep the bill at zero, rely only on what fits inside Termux, and learn where the real bottlenecks hide. What follows is a honest walk‑through of the architecture, the things that broke, and the tiny tricks that kept the whole thing alive.

Termux gives you a Linux‑like environment without root. You can install packages with pkg, run Node.js, Python, or even compile C binaries. For a phone‑only stack it’s the only realistic way to get a full‑featured shell, a package manager, and persistent storage that survives reboots. The downside? You’re stuck with the phone’s RAM (usually 2‑4 GB) and no /tmp directory that behaves like a typical Linux tmpfs. Anything you write to /data/data/com.termux/files/usr/tmp is still counted against the app’s private storage, which can fill up quickly if you’re not careful.

arecord. llama.cpp. The binary is about 1.4 GB; I store it on the external SD card to avoid eating internal storage. The data flow is: mic → Vosk → text → LLM → JSON command → executor → action → feedback (spoken back via espeak-ng).

The first time I launched the Llama binary, the phone killed it after ~12 seconds with “Killed”. Android’s low‑memory killer (LMK) treats any process that exceeds a certain fraction of total RAM as a candidate. With 3 GB RAM, the Llama process plus the Node.js wrapper and Vosk exceeded the limit.

Fix: I swapped the model for a 3‑B parameter quantized version (≈800 MB) and pinned the Llama process to a specific CPU core using taskset. I also lowered the JVM heap for Node ( node --max-old-space-size=256). After those tweaks the LMK left us alone.

Termux’s default shell is bash, but it behaves oddly with quoted heredocs when the delimiter contains spaces. I tried to embed a multi‑line prompt for the LLM like this:

LLM_INPUT=$(cat <<'EOF'
You are a helpful agent. Respond with JSON.
User said: "$(cat /data/data/com.termux/files/home/last_speech.txt)"
EOF
)

The script would hang waiting for the delimiter because Termux’s bash strips the newline before the closing EOF when the heredoc is quoted. The workaround was to avoid quoting the delimiter and instead escape any $ inside the block:

LLM_INPUT=$(cat <<EOF
You are a helpful agent. Respond with JSON.
User said: \"$(cat /data/data/com.termux/files/home/last_speech.txt)\"
EOF
)

That tiny change eliminated the hang.

Because Termux doesn’t mount a real tmpfs, writing large temporary files to /tmp quickly exhausts the app’s private storage. I initially dumped the Vosk audio chunks there, leading to “No space left on device” errors after a few minutes.

Fix: I redirected all temporary files to a folder on the external SD card (/storage/XXXX-XXXX/tmp) and added a periodic cleanup cron job (*/5 * * * * rm -rf /storage/XXXX-XXXX/tmp/*). The external storage is slower but far more capacious.

Below is the Bash watchdog I run in a separate Termux session. It checks each critical process every 15 seconds and restarts it if it’s missing or has exceeded a memory threshold (using pmap to gauge RSS).

#!/data/data/com.termux/files/usr/bin/bash
while true; do
  if ! pgrep -f "vosk_listener.js" >/dev/null; then
    echo "$(date): Vosk listener dead, restarting"
    node /data/data/com.termux/files/home/vosk_listener.js &
  fi

  if ! pgrep -f "llama.cpp" >/dev/null; then
    echo "$(date): LLM backend dead, restarting"
    taskset -c 0-3 /data/data/com.termux/files/home/llama.cpp/main -m /sdcard/models/llama-2-7b-q4.bin -n 128 &
  fi

  for pid in $(pgrep -f "llama.cpp"); do
    rss=$(pmap $pid | tail -1 | awk '{print $2}' | sed 's/K$//')
    if [ $rss -gt 800000 ]; then
      echo "$(date): LLM RSS $rss KB > limit, killing $pid"
      kill $pid
    fi
  done

  sleep 15
done

Save this as watchdog.sh, make it executable ( chmod +x watchdog.sh), and launch it with ./watchdog.sh &. The loop writes timestamps to the terminal, which you can pipe to a log file if you like.

Here’s the quoted heredoc I use to build the prompt that gets fed to the LLM. It pulls the latest transcription, inserts it into a static instruction block, and then pipes the result to the Llama binary.

bash
PROMPT=$(cat <<'EOF'
You are an AI agent running on a phone. 
When the user speaks, output a JSON object with keys "action" and "params".
Valid actions: "send_sms", "toggle_wifi", "open_app", "speak".
Only output JSON, no extra text.

User said: "$(cat /data/data/com.termux/files/home/last_speech.txt)"
EOF

---
The toolkit from this journey: https://samhiotisiddn-jpg.github.io/ironvision-store/