{"slug": "r-457-a-27m-parameter-reasoning-model-running-offline-across-two-esp32-s3", "title": "R-457 – a 27M-parameter reasoning model running offline across two ESP32-S3", "summary": "A 27-million-parameter reasoning model that runs entirely offline on two ESP32-S3 microcontrollers, costing about $40 in hardware, achieves 85.5% accuracy on unseen comparative adjectives, up from 63.5% in the previous version, according to the project's developer. The model, which operates at 0.25–0.42 tokens per second, can reason over facts, use on-chip tools for arithmetic and counting, and honestly refuse questions when facts are insufficient.", "body_md": "A 27-million-parameter reasoning model that runs entirely offline on two ESP32-S3 microcontrollers. No WiFi, no cloud, no phone. It answers questions from a knowledge bank on an SD card, calls tools on the chip for arithmetic it cannot do reliably in its head, refuses honestly when the facts do not support an answer, and learns new facts you teach it at runtime.\n\nIt is slow — about 0.3 tokens per second. That is the honest headline. What\nit demonstrates is not speed but that a model this small can be made to\n*reason over supplied facts and know when to say no*, on hardware that costs\nabout $40 total.\n\n```\n> /ask What is the atomic mass of silicon?\n[retrieved 2 fact(s) from the bank]\n\nFacts: The atomic mass of silicon is 28.085 atomic mass units.\n       The atomic number of silicon is 14.\nQuestion: What is the atomic mass of silicon?\nReasoning: Looking at the facts, the atomic mass of silicon is 28.085\n           atomic mass units. So the answer is 28.085 atomic mass units.\nAnswer: 28.085 atomic mass units\n\n[52 tokens in 156.0s — 0.33 tok/s, conf avg 0.98 min 0.47]\n```\n\nNote the trap in that example: the atomic *number* (14) sits right next to\nthe atomic *mass* (28.085), and the model does not confuse them. Most of the\nwork in this project went into that kind of discrimination.\n\nEvery capability below is verified running on the hardware, not just in training metrics.\n\n**Reasoning**\n\n- Transitive chains (\"A is taller than B, B is taller than C…\")\n- Syllogisms\n- Negation\n- Refusal when the question is underdetermined (\"both compared to a third thing, but not to each other — cannot be determined\")\n\n**Tools** — the chip computes, the model routes\n\n`<calc>`\n\narithmetic: the model writes`<calc>55/5=`\n\n, the firmware computes 11 and feeds it back as forced tokens, so the KV cache stays consistent`<count>`\n\nletter counting:`<count>strawberry,r=`\n\n→ 3\n\n**Knowledge**\n\n- 792-key knowledge bank on the SD card, binary-searched in ~11 reads\n- Extractive lookup with distractor facts present\n- Honest refusal when the retrieved facts do not contain the answer\n- Synonym expansion so \"how heavy\" also finds \"density\" cards\n\n**Generation**\n\n`<write>`\n\nmode: grounded paragraphs that use only the supplied numbers\n\n**Runtime behaviour**\n\n`/learn`\n\n— teach it a fact; written to the SD card, used immediately`/refused`\n\n— every question it could not answer, logged for later teaching`/selftest`\n\n— runs its own canaries and prints PASS/FAIL- Confidence flagging — prints per-answer confidence, warns when low\n- Boot fingerprint — verifies the tokenizer encodes correctly at startup\n\nHeld-out accuracy, 200 examples per set, on-device tool execution:\n\n| set | ft7b | previous (ft6) |\n|---|---|---|\n| axis transfer (unseen adjectives) | 85.5% |\n63.5% |\narithmetic (`<calc>` ) |\n97.0% | 97.0% |\ncounting (`<count>` ) |\n99.0% | 92.0% |\n| Forth stack tracing | 98.5% | 98.0% |\n| Lisp expression tracing | 100.0% | 100.0% |\n| physics word problems | 99.5% | 100.0% |\n| mixed in-distribution | 89.0% | 89.0% |\n| reworded phrasings | 93.0% | 89.0% |\n| 4-item chains | 68.5% | 55.0% |\n| lookup | 61.0% | 57.5% |\n| lookup (single fact) | 62.5% | 67.5% |\n| lookup v2 | 57.0% | 60.0% |\n\n**Axis transfer** is the result worth explaining. The model is trained on\ncomparative adjectives like \"taller\" and \"heavier\". At evaluation it is\ngiven adjectives it has *never seen* — \"sharper\", \"richer\", \"deeper\" — and\nmust still chain them correctly. 85.5% means the transitive relation\ngeneralised past the specific words it was taught on.\n\nThe jump from 63.5% to 85.5% is most likely explained by fixing a silent\ndata-corruption bug in the training data loader rather than by the training\ndata changes made at the same time — the two were not isolated, and the\ncontrolled comparison was never run. See `FAILURES.md`\n\n#1; it is the most\nuseful thing in this repo.\n\n**Speed:** 0.25–0.42 tok/s across the two boards. Measured, not estimated.\nThe UART link between boards costs about 2% of that; the rest is compute.\n\n| part | role | cost |\n|---|---|---|\n| Guition JC3248W535C (ESP32-S3, 16MB flash, 8MB PSRAM) | worker — layers 0–3 | ~$15 |\n| Waveshare ESP32-S3-Touch-LCD-4.3 (16MB / 8MB) | head — embedding, layers 4–7, classifier, tokenizer, SD | ~$25 |\n| microSD card (any size; 512MB is plenty) | knowledge bank, learned facts, refusal log | — |\n| 3 jumper wires | the link | — |\n\n**Why two boards:** the INT4 model image is 17.2MB. A 16MB flash chip cannot\nhold it alongside the firmware — the largest usable model partition is about\n14MB. Splitting the layers across two boards solves this.\n\nTwo boards buy **capacity, not speed**. Generation is sequential: while one\nboard computes its layers the other waits. The split makes the model\n*possible*, not faster.\n\n**Wiring** (three wires, boards powered off):\n\n```\nhead GPIO8  ──────→ worker GPIO18     (head TX → worker RX)\nhead GPIO9  ←────── worker GPIO17     (head RX ← worker TX)\nhead GND    ────────  worker GND\n```\n\n460800 baud, CRC + retry. The head uses GPIO 8/9 because on the Waveshare board GPIO 17/18 are wired to the LCD panel and unavailable — 8/9 are the I2C terminal block, used here as a plain UART.\n\n- Download\n`r457_ft7b_worker.bin`\n\nand`r457_ft7b_head.bin`\n\nfrom[Releases](/harmansingh4163-ai/R-457/releases). - Flash the model images:\n\n```\nesptool.py --chip esp32s3 --port <guition-port>   write_flash 0x1F0000 r457_ft7b_worker.bin\nesptool.py --chip esp32s3 --port <waveshare-port> write_flash 0x1F0000 r457_ft7b_head.bin\n```\n\n- Copy\n`kb.bin`\n\nto the root of the SD card, insert it in the Waveshare. - Open\n`sketches/pipeline_worker`\n\nand`sketches/pipeline_head`\n\nin the Arduino IDE and upload each to its board. Board settings and the two board-specific gotchas are in`BUILD.md`\n\n— read it, they are not obvious. - Wire the three jumpers, power the worker first, then the head.\n- Open the serial monitor on the head at 115200 and type\n`/selftest`\n\n.\n\nExpect:\n\n```\nkb: ready, 792 keys (SD)\nboot check: mode-token ok, newline ok\nReady: emb + 4 local layers of 8 total.\nSELFTEST: PASS (silicon ok, aluminium ok)\n```\n\nFull build-from-source instructions — training, export, splitting — are in\n`BUILD.md`\n\n.\n\n**Prompt format.** Everything the model sees is four parts:\n\n```\n<reason>\nFacts: The density of aluminium is 2700 kilograms per cubic metre.\nQuestion: What is the density of aluminium?\nReasoning: The facts give the density of aluminium as 2700 kilograms per\n           cubic metre. So the answer is 2700 kilograms per cubic metre.\nAnswer: 2700 kilograms per cubic metre\n```\n\n`<reason>`\n\nand `<write>`\n\nare single reserved tokens that switch the model's\nmode. Facts come from the SD bank, from what you typed, or from what you\nhave taught it with `/learn`\n\n.\n\n**Tools.** The model is not trusted with arithmetic. It writes the *call*\nand stops at the `=`\n\n; the chip computes the answer and injects it back as\nforced tokens. Generation continues with a KV cache that never saw a wrong\nnumber. A 27M model cannot divide reliably, but it can learn to ask.\n\n**Refusal.** Roughly a third of training examples are questions the facts\ndo not answer. The model is taught to name what is missing and stop. This is\nwhat makes the knowledge bank usable — a lookup system that confabulates\nwhen it misses is worse than no lookup at all.\n\n**The learning loop.** Ask something it cannot answer → it refuses and logs\nthe question to `/refused.txt`\n\n→ you check `/refused`\n\nlater → you teach it\nwith `/learn The boiling point of silicon is 3265 degrees Celsius.`\n\n→ it\nanswers correctly from then on. Verified end to end on hardware, no\nretraining involved.\n\n```\ncore/         llm_core.c/h    — inference, INT4 matmul, tokenizer, KV cache\n              kb.cpp/h        — knowledge bank reader, retrieval, prompt build\nsketches/     pipeline_head/  — the user-facing board (SD, tools, commands)\n              pipeline_worker/— the coprocessor board\npc_tools/     construct.py    — training data generator\n              build_kb.py     — knowledge bank builder\n              export_model.py — checkpoint → INT4 device image\n              split_image.py  — device image → two board images\n              eval_fixed.py   — fixed-yardstick loss evaluation\n              evaluate_tools.py — held-out capability scoring with tools\ndocs/         FAILURES.md, ROADMAP.md, BUILD.md\n```\n\nIf you are here to build one: `BUILD.md`\n\n.\n\nIf you are here because you are training small models yourself:\n** FAILURES.md first.** It documents a silent data-corruption bug that\ntaxed every training run in this project for weeks, two tokenizer bugs that\nproduced convincing-looking garbage, and four capability experiments that\nfailed with their diagnoses. That file cost far more to produce than this\none.\n\nMIT. Built on [llama2.c](https://github.com/karpathy/llama2.c) by Andrej\nKarpathy. TinyStories dataset by Eldan & Li. Per-layer-embedding experiments\ninformed by [slvDev/esp32-ai](https://github.com/slvDev/esp32-ai).", "url": "https://wpnews.pro/news/r-457-a-27m-parameter-reasoning-model-running-offline-across-two-esp32-s3", "canonical_source": "https://github.com/harmansingh4163-ai/R-457", "published_at": "2026-08-03 06:27:54+00:00", "updated_at": "2026-08-03 06:52:38.612508+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research", "ai-tools"], "entities": ["ESP32-S3", "R-457"], "alternates": {"html": "https://wpnews.pro/news/r-457-a-27m-parameter-reasoning-model-running-offline-across-two-esp32-s3", "markdown": "https://wpnews.pro/news/r-457-a-27m-parameter-reasoning-model-running-offline-across-two-esp32-s3.md", "text": "https://wpnews.pro/news/r-457-a-27m-parameter-reasoning-model-running-offline-across-two-esp32-s3.txt", "jsonld": "https://wpnews.pro/news/r-457-a-27m-parameter-reasoning-model-running-offline-across-two-esp32-s3.jsonld"}}