{"slug": "running-an-autonomous-ai-agent-stack-on-android-with-termux-no-cloud-just-a", "title": "Running an Autonomous AI Agent Stack on Android with Termux — No Cloud, Just a Phone", "summary": "A developer documented building a fully offline AI agent stack on an old Android phone using Termux, chaining Vosk speech recognition, a quantized llama.cpp model, a JSON command executor, and espeak-ng for spoken feedback. The writeup details Android low-memory-killer kills of the 1.4 GB Llama binary, fixed by dropping to a ~800 MB 3B quantized model, pinning the process with taskset, and capping Node's heap at 256 MB, plus workarounds for Termux heredoc hangs and the lack of a real tmpfs. A Bash watchdog restarts dead processes every 15 seconds and kills the LLM backend when RSS exceeds 800 MB.", "body_md": "I’ve spent the last few months trying to turn an old Android phone into a self‑contained AI agent that can listen, reason, and act without ever reaching for a cloud service. The goal was simple: keep the bill at zero, rely only on what fits inside Termux, and learn where the real bottlenecks hide. What follows is a honest walk‑through of the architecture, the things that broke, and the tiny tricks that kept the whole thing alive.\n\nTermux gives you a Linux‑like environment without root. You can install packages with `pkg`, run Node.js, Python, or even compile C binaries. For a phone‑only stack it’s the only realistic way to get a full‑featured shell, a package manager, and persistent storage that survives reboots. The downside? You’re stuck with the phone’s RAM (usually 2‑4 GB) and no `/tmp` directory that behaves like a typical Linux tmpfs. Anything you write to `/data/data/com.termux/files/usr/tmp` is still counted against the app’s private storage, which can fill up quickly if you’re not careful.\n\n`arecord`.` llama.cpp`. The binary is about 1.4 GB; I store it on the external SD card to avoid eating internal storage.\nThe data flow is: mic → Vosk → text → LLM → JSON command → executor → action → feedback (spoken back via `espeak-ng`).\n\nThe first time I launched the Llama binary, the phone killed it after ~12 seconds with “Killed”. Android’s low‑memory killer (LMK) treats any process that exceeds a certain fraction of total RAM as a candidate. With 3 GB RAM, the Llama process plus the Node.js wrapper and Vosk exceeded the limit.\n\n**Fix:** I swapped the model for a 3‑B parameter quantized version (≈800 MB) and pinned the Llama process to a specific CPU core using `taskset`. I also lowered the JVM heap for Node (` node --max-old-space-size=256`). After those tweaks the LMK left us alone.\n\nTermux’s default shell is `bash`, but it behaves oddly with quoted heredocs when the delimiter contains spaces. I tried to embed a multi‑line prompt for the LLM like this:\n\n```\nLLM_INPUT=$(cat <<'EOF'\nYou are a helpful agent. Respond with JSON.\nUser said: \"$(cat /data/data/com.termux/files/home/last_speech.txt)\"\nEOF\n)\n```\n\nThe script would hang waiting for the delimiter because Termux’s `bash` strips the newline before the closing `EOF` when the heredoc is quoted. The workaround was to avoid quoting the delimiter and instead escape any `$` inside the block:\n\n```\nLLM_INPUT=$(cat <<EOF\nYou are a helpful agent. Respond with JSON.\nUser said: \\\"$(cat /data/data/com.termux/files/home/last_speech.txt)\\\"\nEOF\n)\n```\n\nThat tiny change eliminated the hang.\n\nBecause Termux doesn’t mount a real tmpfs, writing large temporary files to `/tmp` quickly exhausts the app’s private storage. I initially dumped the Vosk audio chunks there, leading to “No space left on device” errors after a few minutes.\n\n**Fix:** I redirected all temporary files to a folder on the external SD card (`/storage/XXXX-XXXX/tmp`) and added a periodic cleanup cron job (`*/5 * * * * rm -rf /storage/XXXX-XXXX/tmp/*`). The external storage is slower but far more capacious.\n\nBelow is the Bash watchdog I run in a separate Termux session. It checks each critical process every 15 seconds and restarts it if it’s missing or has exceeded a memory threshold (using `pmap` to gauge RSS).\n\n``` bash\n#!/data/data/com.termux/files/usr/bin/bash\nwhile true; do\n  # Check Vosk listener\n  if ! pgrep -f \"vosk_listener.js\" >/dev/null; then\n    echo \"$(date): Vosk listener dead, restarting\"\n    node /data/data/com.termux/files/home/vosk_listener.js &\n  fi\n\n  # Check LLM backend\n  if ! pgrep -f \"llama.cpp\" >/dev/null; then\n    echo \"$(date): LLM backend dead, restarting\"\n    taskset -c 0-3 /data/data/com.termux/files/home/llama.cpp/main -m /sdcard/models/llama-2-7b-q4.bin -n 128 &\n  fi\n\n  # Simple OOM guard: if RSS > 800M, kill and respawn\n  for pid in $(pgrep -f \"llama.cpp\"); do\n    rss=$(pmap $pid | tail -1 | awk '{print $2}' | sed 's/K$//')\n    if [ $rss -gt 800000 ]; then\n      echo \"$(date): LLM RSS $rss KB > limit, killing $pid\"\n      kill $pid\n    fi\n  done\n\n  sleep 15\ndone\n```\n\nSave this as `watchdog.sh`, make it executable (` chmod +x watchdog.sh`), and launch it with `./watchdog.sh &`. The loop writes timestamps to the terminal, which you can pipe to a log file if you like.\n\nHere’s the quoted heredoc I use to build the prompt that gets fed to the LLM. It pulls the latest transcription, inserts it into a static instruction block, and then pipes the result to the Llama binary.\n\n```\nbash\nPROMPT=$(cat <<'EOF'\nYou are an AI agent running on a phone. \nWhen the user speaks, output a JSON object with keys \"action\" and \"params\".\nValid actions: \"send_sms\", \"toggle_wifi\", \"open_app\", \"speak\".\nOnly output JSON, no extra text.\n\nUser said: \"$(cat /data/data/com.termux/files/home/last_speech.txt)\"\nEOF\n\n---\nThe toolkit from this journey: https://samhiotisiddn-jpg.github.io/ironvision-store/\n```\n\n", "url": "https://wpnews.pro/news/running-an-autonomous-ai-agent-stack-on-android-with-termux-no-cloud-just-a", "canonical_source": "https://dev.to/sam_hiotis_117598dbfa3ac2/running-an-autonomous-ai-agent-stack-on-android-with-termux-no-cloud-just-a-phone-33dk", "published_at": "2026-09-25 14:08:41+00:00", "updated_at": "2026-09-25 14:31:24.661897+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-tools", "mlops"], "entities": ["Termux", "llama.cpp", "Vosk", "espeak-ng", "Node.js", "Android"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/running-an-autonomous-ai-agent-stack-on-android-with-termux-no-cloud-just-a", "markdown": "https://wpnews.pro/news/running-an-autonomous-ai-agent-stack-on-android-with-termux-no-cloud-just-a.md", "text": "https://wpnews.pro/news/running-an-autonomous-ai-agent-stack-on-android-with-termux-no-cloud-just-a.txt", "jsonld": "https://wpnews.pro/news/running-an-autonomous-ai-agent-stack-on-android-with-termux-no-cloud-just-a.jsonld"}}