cd /news/large-language-models/local-llm-on-esp32-s3-control-a-serv… · home › topics › large-language-models › article
[ARTICLE · art-170639] src=hackster.io ↗ pub= topic=large-language-models verified=true sentiment=↑ positive

Local LLM on ESP32-S3 - Control a Servo and RGB LED

A Visuino tutorial demonstrates running Needle 2, a 45M-parameter tool-calling model from Cactus Compute, directly on an ESP32-S3 board with 16 MB flash and 8 MB Octal/OPI PSRAM, where the model interprets commands such as "move servo to 90 degrees" and returns structured tool calls that C++ firmware validates before driving a servo or the onboard WS2812 RGB LED on GPIO48. The reference port memory-maps the model from flash and uses PSRAM for working memory, and the project reports roughly 47 seconds for a reasoning-on request on its target board, with actual time varying by prompt and output length. The tutorial requires Arduino-ESP32 core 3.3.10, Visuino Professional with the Custom Code component, Chrome or Edge for the local model flasher, and a custom partitions.csv reserving flash from 0x210000 for the model.

by read16 min views1 publishedNov 16, 2026
Local LLM on ESP32-S3 - Control a Servo and RGB LED
Image: Hackster (auto-discovered)

Can an ESP32-S3 run a real language model locally? Yes. In this tutorial we run Needle 2, a compact 45M-parameter tool-calling model, directly on an ESP32-S3 with 16 MB flash and 8 MB Octal/OPI PSRAM.

You will type commands such as:

move servo to 90 degrees

blink blue 5 times

move servo to 90 degrees then blink blue 3 times

The sentence goes directly to Needle 2. The Arduino code does not search the sentence for words such as “servo”, “blue”, or “90”. Needle interprets the request and returns structured tool calls. The supplied C++ then validates those structured calls and either sends a normalized value to a Visuino Servo component or controls the onboard RGB LED.

Needle 2 is an open 45M-parameter model from Cactus Compute designed for tool calling, device use, and structured extraction. The official model is distributed as a compact.cact binary. The ESP32-S3 port used by this tutorial memory-maps the model from flash and uses PSRAM for working memory.

This makes Needle especially suitable for embedded control. It is not being used here as a general chatbot. Its job is to convert a human request into a small, structured action that firmware can validate and execute.

The reference ESP32-S3 port targets:

  • ESP32-S3, dual core, 240 MHz.
  • 16 MB flash .
  • 8 MB Octal PSRAM .
  • An N16R8-class memory configuration.
  • WS2812-compatible onboard RGB LED on GPIO48 on the reference/tested board.

The reference project reports roughly ~47 seconds for a reasoning-on request on its target board; actual time varies with prompt/output length. Long or complex requests can take substantially longer.

  • ESP32-S3 board with 16 MB flash + 8 MB Octal/OPI PSRAM .

  • USB data cable.

  • Small hobby servo.

  • Suitable servo power source, commonly 5 V for small hobby servos.

  • Jumper wires and optional breadboard.

  • Visuino Professional / a Visuino edition with the Custom Code component.

  • Arduino-ESP32 core. This project was tested with 3.3.10 .

  • Desktop web browser Google Chrome or Microsoft Edge for the attached local model flasher.

  • The supplied Needle2_ESP32S3_Visuino_Files.zip package.

Visuino demo project (Needle2.visuino) file is also available below

Step 3: Install the Needle2 Arduino Library Extract the complete ZIP Needle2_ESP32S3_Visuino_Files.zip.

Copy this folder:

Libraries\Needle2

into:

C:\Users\<YOUR_USER>\Documents\Arduino\libraries\Needle2

After copying, this should exist:

C:\Users\<YOUR_USER>\Documents\Arduino\libraries\Needle2\src\Needle2.h

and:

C:\Users\<YOUR_USER>\Documents\Arduino\libraries\Needle2\src\Needle2UserTools.h

Step 4: Configure the Generated Sketch Folder and Copy Partitions.csv Before we build and upload the project, we need to prepare the custom partition file used by Needle 2.

In Visuino, open:

Arduino → Configure

Under Generated sketch directory, check that the path is:

%userprofile%\Documents\Visuino\Generated

If this path is already shown, leave it unchanged.

Now open Windows File Explorer and navigate to:

%userprofile%\Documents\Visuino\Generated

If the Generated folder does not exist yet, create it manually.

Next, open the downloaded tutorial files and locate:

Visuino\partitions.csv

Copy partitions.csv into the Generated folder.

Before the first build, the folder may contain only:

Generated\

partitions.csv

Later, when Visuino generates the Arduino sketch, it will also create:

Generated\

Generated.ino

partitions.csv

The filename must be exactly:

partitions.csv

Make sure Windows has not renamed it to partitions.txt or partitions.csv.txt.

This custom partition file reserves the flash area used by the Needle 2 model.

The important model partition begins at:

0x210000

You only need to copy partitions.csv once, as long as you continue using the same Generated sketch directory.

Do not upload the project yet. We will first finish configuring the Visuino components, then compile and upload everything in the following steps.

Step 5: Start Visuino, and Select the ESP32-S3 Board Type Open Visuino Professional and select:

Board: ESP32 S3 Development Kit C-1

Step 6: In Visuino Add Custom Code and Servo Component Add these components:

  • Custom Code
  • Servo

The RGB LED does not need a separate Visuino component in this V26 project; it is controlled by the supplied code.

Step 7: Parse the V26 Arduino Code Now we will import the supplied Needle 2 Arduino code into the Custom Code component.

Right-click on the CustomCode1 component.

From the popup menu, select:

Parse Arduino Code

A window will open where you can paste Arduino code.

Open the supplied file in Arduino IDE or notepad:

Visuino\Needle2_Visuino_LED_SERVO_V26.ino

Copy the complete contents of the file and paste them into the Parse Arduino Code window.

Click OK.

Visuino will analyze the Arduino code and another window will open showing the detected code sections and settings.

Click OK again.

Visuino will now automatically configure the Custom Code component with the required sections, including:

  • Includes
  • Global Declarations
  • Setup code
  • Loop code
  • Helper functions and other parsed code sections

After this step, the Needle 2 code is imported into CustomCode1 and is ready to be used in the project.

Right-click on the CustomCode1 component.

From the popup menu, select:

Edit Outputs

In the Outputs window drag "Analog" element to the left side

Close the Outputs window

Connect:

CustomCode1.Analog1 → Servo1.In

Connect Servo1.Out to the ESP32-S3 pin you want to use for the servo signal. The tested example can use GPIO17.

Example physical wiring:

Servo signal → ESP32 GPIO17 pin

Servo V+ → ESP32 5V pin

Servo GND → supply GND

Do not power a normal hobby servo from the ESP32 3.3 V output.

Step 10: In Visuino Upload the Project On the bottom tab select Build and set this settings:

Upload Speed: 921600

USB Mode: Hardware CDC and JTAG

USB CDC On Boot: Enabled

Upload Mode: UART0 / Hardware CDC

CPU Frequency: 240MHz (WiFi)

Flash Mode: QIO 80MHz

Flash Size: 16MB (128Mb)

Partition Scheme: No OTA (2MB APP/2MB SPIFFS)

PSRAM: OPI PSRAM

Arduino Runs On: Core 1

Events Run On: Core 1

Erase All Flash Before Sketch Upload: Disabled

Make sure the correct port is selected, then click on the "Compile/Build and Upload" button.

The model is flashed separately from the Visuino firmware. The package includes a local browser flasher, so users do not need Python, esptool commands, BAT files, or a custom board menu.

First close the Visuino Serial Monitor so COM port access is free.

Open this folder:

Needle2_Local_Flasher\

Keep these files together:

index.html

needle2.cact

Then:

  • Open index.html in desktop Chrome or Edge .
  • Click 1. Connect ESP32-S3 .
  • Choose the ESP32-S3 serial port.
  • Click 2. Flash Needle 2 .
  • Wait until the progress reaches 100% and the page says the flash completed successfully.

The flash address is hard-coded in the page:

0x210000

The user does not type the address.

The page also checks the tested model file before flashing:

Size: 13, 737, 807 bytes

SHA256: b43aabfcaf1a6db6acf488076eab71d823c08697c7af4521fc1d174b60ede5ba

It requests no full-chip erase. On the hardware used while preparing this tutorial, flashing the 13.7 MB model took about 7 minutes; other computers/USB connections may be faster or slower.

The current tested HTML loads Espressif esptool-js 0.6.1 from a CDN, so the PC needs an Internet connection while opening the flasher. The needle2.cact file itself stays local and is sent from the browser directly to the connected ESP32-S3.

If Chrome blocks automatic of the local needle2.cact, the page displays a Choose needle2.cact button. Select the copy in the same flasher folder and continue.

Now we can start the project and check if Needle 2 is running correctly.

Return to Visuino and click the Play button.

At the bottom of the Visuino window, open the Serial tab.

Make sure the correct ESP32-S3 COM port is selected.

For this project, use the same port that was used during upload.

Set the Serial Monitor baud rate to:

115200

Then connect/open the serial port.

Once the Serial Monitor is connected, press the RESET / EN button on the ESP32-S3 board once.

This restarts the board so you can see the complete startup messages in the Serial Monitor.

Wait a few seconds while the ESP32-S3 starts and initializes Needle 2.

If everything is configured correctly, you should see output similar to:

=== Needle 2 + Visuino FINAL SIMPLE LED + SERVO V26 ===

Tools: set_servo + blink_led

Servo output: Analog1

Reasoning: ON

Plan execution: ATOMIC PREFLIGHT

Starting Needle 2...

model mapped: 13762560 bytes

Needle 2 is ready.

The most important message is:

Needle 2 is ready.

This confirms that the Needle 2 model was found and opened correctly and that the ESP32-S3 is ready to receive natural-language commands.

Do not send a command until you see:

Needle 2 is ready.

In the next step, we will send our first command and control the servo.

Step 13: Test RGB LED Control

In the Serial Monitor, type the following command and press Enter, or click the Send button:

blink blue 5 times

After Needle 2 processes the request, the RGB LED on the ESP32-S3 should blink blue 5 times.

Next, try setting how long the LED should blink and how fast:

blink red for 20s 2hz

The LED should blink red for 20 seconds at 2 Hz.

You can use these LED colors:

red, green, blue, yellow, purple, white

You can tell Needle 2 how many times to blink the LED, for example:

blink red 5 times

or how long to blink it:

blink red for 20 seconds

You can also specify the blinking speed:

blink red for 20 seconds at 2hz

Before any hardware action is performed, the program checks that the generated values are valid and within the allowed limits.

Tip: For the most reliable results, use short, clear commands written in lowercase, like the examples above.

In the Serial Monitor, type:

move servo to 30 degrees

and press Enter, or click the Send button.

Needle 2 should generate a set_servo command with the value:

degrees: 30

Visuino then converts the angle into the normalized value used by the Servo component:

30 / 180 = 0.1667

The servo should move to approximately 30 degrees.

You can also try:

move servo to 0 degrees
move servo to 90 degrees
move servo to 180 degrees

Before moving the servo, the program checks that the requested angle is a valid whole number between 0 and 180 degrees.

Tip: For the most reliable results, use short, clear commands written in lowercase.

Now try controlling both the servo and the RGB LED with one command:

move servo to 90 degrees then blink blue 3 times

Needle 2 should create two actions in the correct order:

Move the servo to 90 degrees.Blink the RGB LED blue 3 times.

Before anything is executed, the program checks the complete set of generated actions.

If one of the actions contains an invalid or unsafe value, the whole command is blocked and no hardware action is performed.

For the best results, keep combined commands short and simple.

Needle 2 is a small language model running directly on the ESP32-S3, so shorter commands are usually more reliable than long or complicated instructions.

Step 16: Practical Limits Needle 2 works surprisingly well on the ESP32-S3, but it is still a very small language model running on a microcontroller.

For the best results:

  • Use one action at a time, or a short two-action command.
  • Avoid long or complicated instructions.
  • Use clear values and units, for example: 20s and 2hz.
  • Keep the number of available tools small.
  • Give Needle 2 time to process the request. Some commands can take tens of seconds.
  • Remember that the model can sometimes misunderstand a request.

The program checks the generated commands before controlling the hardware. If the result is invalid or outside the allowed limits, the action is blocked.

This combination works well for embedded AI:

Needle 2 understands the request, while the program checks and safely executes the hardware action.

The Needle 2 model normally needs to be flashed only once.

For future Visuino changes:

  • Keep partitions.csv in:
%userprofile%\Documents\Visuino\Generated
  • Keep Erase All Flash Before Sketch Upload = Disabled .
  • Build and upload the Visuino project normally.

You do not need to run the HTML model flasher again unless you erase the flash, replace the model, or Needle 2 can no longer find the model.

Normal firmware uploads do not overwrite the Needle 2 model stored at 0x210000.

Step 18: Troubleshooting and Getting Help TIf something does not work, first check the Serial Monitor output and the Visuino build settings.

Make sure Visuino is configured with:

PSRAM: OPI PSRAM

Then build and upload the project again.

nd_model_open failed (-2) If you see something similar to:

ERR nd_model_open failed (-2)

the ESP32-S3 found the model partition, but the Needle 2 model is missing or was not flashed correctly.

Close the Visuino Serial Monitor and run the supplied HTML model flasher again.

Do not erase the complete flash.

Check that the generated sketch directory is:

%userprofile%\Documents\Visuino\Generated

and that this folder contains:

partitions.csv

The file must be there before building the project.

Analog1 was not declared Make sure the Custom Code component contains an Analog output named exactly:

Analog1

Then parse the supplied V26 Arduino code again.

Servo does not move Check:

  • CustomCode1.Analog1 is connected to Servo1.In.
  • The correct servo output pin is selected.
  • The servo has a suitable power supply.
  • The servo and ESP32-S3 share a common ground.

Then test:

move servo to 90 degrees

RGB LED does not work This tutorial uses the onboard addressable RGB LED on GPIO48 of the tested ESP32-S3 board.

Some ESP32-S3 boards use a different LED pin or different LED hardware.

Browser flasher cannot connect

Close the Visuino Serial Monitor and any other program using the ESP32-S3 COM port.

Then reconnect the USB cable, reload index.html in desktop Chrome or Edge, and try again.

I erased the complete flash Upload the Visuino project again and then run the Needle 2 HTML model flasher again.

Afterward, remember to keep:

Erase All Flash Before Sketch Upload: Disabled

Use short, clear commands written in lowercase.

For example:

move servo to 90 degreesblink blue 5 times

Needle 2 is a small embedded model and can sometimes misunderstand long or complicated requests.

The V26 program checks the generated actions before controlling the hardware, so invalid actions can be blocked instead of executed.

How to Ask ChatGPT for Help If you still cannot solve the problem, provide ChatGPT with:

  • the complete build/upload log
  • the complete Serial Monitor output
  • a screenshot of the Visuino settings or wiring if relevant

You can use this prompt:

I am following the ESP32-S3 + Needle 2 + Visuino V26 tutorial.

Hardware:

  • ESP32-S3
  • 16 MB flash
  • 8 MB OPI/Octal PSRAM

Visuino settings:

  • Flash Size: 16MB
  • Partition Scheme: No OTA
  • PSRAM: OPI PSRAM
  • Erase All Flash Before Sketch Upload: Disabled
  • Generated sketch directory: %userprofile%\Documents\Visuino\Generated

The supplied partitions.csv is in the Generated folder.

The Needle 2 model was flashed using the supplied HTML flasher.

I am using Needle2_Visuino_LED_SERVO_V26.ino.

Please diagnose the problem from the logs below and give me the smallest thing to check first.

BUILD / UPLOAD LOG:

[paste the complete log]

SERIAL MONITOR OUTPUT:

[paste the complete output]

Step 19: Adding New Needle 2 Tools Safely Once the basic project is working, you can add your own Needle 2 tools.

A new tool needs two parts:

  • A tool definition in:

Needle2UserTools.h

  • Matching validation and hardware-control code in the Arduino program.

The tool definition tells Needle 2:

  • the tool name
  • what the tool does
  • which values it can generate

The Arduino code then checks those values and performs the actual hardware action.

For example, adding a tool definition called:

turn_relay

does not automatically control a relay. You must also add code that recognizes turn_relay, validates its arguments, and safely controls the relay output.

Keep Tools Simple Needle 2 is a small embedded model, so keep tool definitions short and clear.

For best results:

  • Add one new tool at a time.
  • Use simple tool names.
  • Keep the number of arguments small.
  • Use clear limits for numeric values.
  • Validate every generated value before controlling hardware.
  • Keep the total number of tools reasonably small.

After adding a new tool, rebuild and upload the Visuino project and test it with short, simple commands first.

Do not rely only on Needle 2 to decide what is safe. The Arduino code should always remain responsible for checking values before any hardware action is performed.

Step 20: FAQDoes Needle 2 need Wi-Fi while controlling the servo or LED? No. After the model and firmware are installed, inference runs locally on the ESP32-S3. No cloud API or PC inference server is used.

Does the local HTML flasher mean the LLM runs in Chrome? No. Chrome is used only as a convenient one-time ESP32 model flasher. Needle inference runs on the ESP32-S3 afterward.

Does the flasher create the partition table? No. partitions.csv creates the custom model partition during the Arduino/Visuino firmware build. The HTML flasher only writes needle2.cact to 0x210000.

Why select No OTA if we use a custom partition table? It gives Visuino a normal built-in board-menu selection. Arduino-ESP32's build system gives precedence to a partitions.csv placed in the sketch folder, so the supplied CSV becomes the actual partition layout for this project.

Do I need the old Needle2 16MB partition menu entry? No. This updated tutorial intentionally avoids it.

Do I need Python or a BAT file? No for the normal tutorial flow. The supplied local HTML flasher handles the model upload in Chrome/Edge.

Do I have to flash the 13.7 MB model after every code upload? No. Flash it once, then keep full-chip erase disabled. Reflash only after a full erase, a model change, or a model-open failure.

Can I add more tools? Yes, but keep the schema compact and add matching deterministic C++ validation/execution for each new tool.

Step 21: Model License and Redistribution The bundled needle2.cact model is from Cactus Compute's Needle 2 release and is distributed under the Apache License 2.0. Keep the included license and attribution/third-party notice files with redistributed copies of the package/model.

The surrounding tutorial can use a different content license; the model remains under its own Apache-2.0 license.

Step 22: Conclusion You now have a genuine local tool-calling language model on an ESP32-S3 interpreting natural-language requests and controlling hardware through Visuino.

The important architecture is simple:

Needle 2 = language understanding

C++ = bounded validation and execution

Visuino = visual hardware integration

The updated setup also removes the most awkward parts of the older process: there is no custom partition menu to install and no Python/BAT model-flashing command. Users select the standard No OTA option, keep the supplied partitions.csv in the configured Generated sketch directory, and flash the model from a local Chrome/Edge page.

[Read more](<javascript:void(0)>)

── more in #large-language-models 4 stories · sorted by recency
── more on @needle 2 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/local-llm-on-esp32-s…] indexed:0 read:16min 2026-11-16 · —