cd /news/ai-agents/connecting-raspberry-pi-to-llm-agent… · home › topics › ai-agents › article
[ARTICLE · art-170723] src=hackster.io ↗ pub= topic=ai-agents verified=true sentiment=↑ positive

Connecting Raspberry Pi to LLM Agents Through MCP

A developer built a system that uses a Raspberry Pi 3 as an intermediary between an LLM agent and the physical world, exposing sensor data as semantic capabilities through the Model Context Protocol (MCP) rather than giving the agent direct hardware access. An ESP32 board handles low-level acquisition from a BME280 or BMP280 environmental sensor, an AHT21 humidity/temperature sensor, an ENS160 air-quality sensor, an MPU6050 IMU, an optional WS2812B RGB LED ring and a Raspberry Pi camera, connecting to the Pi over USB serial. The design keeps reasoning and planning in the LLM agent while the Raspberry Pi owns device state, sensor interpretation, permissions and hardware integration, so agents ask questions like "What is the temperature?" instead of reading GPIO or I2C registers.

by read10 min views4 publishedNov 16, 2026
Connecting Raspberry Pi to LLM Agents Through MCP
Image: Hackster (auto-discovered)

Large Language Models are very good at working with digital information. They can reason over documents, call APIs, analyze data and invoke software tools. But what happens when we want an LLM agent to know what is happening in the physical world?

Can it know the temperature of a room?

Can it detect whether someone is nearby?

Can it access air-quality information?

Can it trigger a physical response on a device?

This project started from that question.

The architecture I built uses a Raspberry Pi as the connection point between an LLM agent and the physical environment.

An ESP32 handles sensors and low-level hardware acquisition. The Raspberry Pi receives that information, interprets it and exposes semantic capabilities through the Model Context Protocol — MCP.

This video shows the final result:

The resulting architecture is:

Physical World      ↓   Sensors      ↓    ESP32      ↓ USBRaspberry Pi      ↓Semantic Services      ↓ MCP Server      ↓  LLM Agent

The goal is not to give an LLM direct access to hardware.

The goal is to let the Raspberry Pi expose meaningful information about the physical world in a form an AI agent can safely understand and use.

Components For the implementation described here I use:

  • Raspberry Pi (I have used Raspberry Pi 3 but you can use 4 or 5 too)
  • ESP32 development board
  • USB cable between ESP32 and Raspberry Pi
  • BME280 or BMP280 environmental sensor
  • AHT21 humidity/temperature sensor
  • ENS160 air-quality sensor
  • MPU6050 IMU
  • optional WS2812B RGB LED ring
  • Raspberry Pi camera

Examples of MCP-compatible clients can include desktop or development environments built around models such as Claude, ChatGPT or other LLM-based agents. The specific LLM is not important to the architecture.

Step 1 — Connecting an LLM Agent to the Physical World The first design decision was deciding where the intelligence should live. I did not want the LLM agent to become the hardware controller.

For example, I did not want an agent to execute operations such as:

read GPIO 17read I2C register 0x21write raw LED valueopen serial port

Those are implementation details. Instead, I wanted the interaction to look more like this:

What is the temperature?How is the air quality?Is somebody present?What is happening in the room?

This immediately creates an architectural boundary. The AI agent handles:

reasoninginterpretationplanningdecision making

The Raspberry Pi handles:

device statesensor interpretationapplication logicpermissionshardware integration

MCP sits between them. A useful way to think about the architecture is:

LLM Agent    ↓semantic request    ↓Raspberry Pi    ↓physical system

instead of:

LLM Agent    ↓hardware driver

This distinction becomes increasingly important when AI agents move from software into robotics, IoT and other physical systems.

The next problem was hardware acquisition. A Raspberry Pi can communicate directly with I²C sensors, but once the number of peripherals starts growing, I prefer separating real-time hardware handling from the application runtime. For this project I use an ESP32 as a dedicated controller. It manages sensors including:

BME280 / BMP280AHT21ENS160MPU6050

and can also control devices such as a WS2812B RGB LED ring. The ESP32 is connected physically to the Raspberry Pi through USB. From the Raspberry Pi point of view, this becomes a serial communication channel. The hardware architecture is therefore:

BME280 ──┐AHT21  ──┤ENS160 ──┤MPU6050 ─┼── ESP32 ── USB Serial ── Raspberry PiLED Ring ─┘

This separation gives each device a clear responsibility.

ESP32 responsibilities The ESP32 handles:

  • environmental sensor acquisition
  • air-quality measurements
  • acceleration and angular velocity
  • RGB LED execution
  • controller heartbeat
  • communication with Raspberry Pi

The Raspberry Pi therefore does not need to continuously manage every low-level sensor operation itself.

Sensor data The environmental layer can provide values such as:

temperaturepressurehumidity

The ENS160 provides air-quality information including:

eCO2TVOCAQI

The MPU6050 provides:

accelerationangular velocity

These capabilities are part of the current controller-backed sensor architecture. But simply transmitting a number is not enough. Real sensors have states. They may be warming up. Data may be stale. Measurements may be technically available but not yet reliable.

For this reason the application distinguishes concepts such as:

measurement acquiredmeasurement freshmeasurement validmeasurement usable

That distinction is especially important when an LLM will eventually reason over the data. You do not want an AI agent confidently telling you that the air quality is poor because it received a value from a sensor that has not completed its initialization cycle.

Communication between ESP32 and Raspberry Pi The ESP32 sends structured information to the Pi through the USB serial connection. In my implementation, communication uses a newline-delimited JSON protocol.

A simplified message could look like:

{  "type": "environment",  "temperature": 23.8,  "humidity": 47.2,  "pressure": 1014.1}

Another message could represent motion:

{  "type": "motion",  "acceleration": {    "x": 0.02,    "y": -0.01,    "z": 0.98  }}

The Raspberry Pi receives these messages and updates its internal view of the physical environment. The ESP32 knows how to read the hardware. The Raspberry Pi knows what the measurements mean to the application. That boundary is one of the most useful parts of the architecture.

Once the Raspberry Pi has access to sensor information, the next step is making that information available to an LLM agent. This is where MCP becomes useful. Instead of designing a custom API specifically for one AI model, the Raspberry Pi exposes an MCP server. The important part is that MCP is not another hardware runtime. It is an adapter over the Raspberry Pi application.

The architecture becomes:

LLM Agent      ↓     MCP      ↓Raspberry Pi MCP Server      ↓Application Services      ↓Semantic Physical Context      ↓ESP32 / Camera / Audio / Hardware

The MCP server runs inside the existing Raspberry Pi runtime instead of creating a second copy of the physical application. This means:

one Raspberry Pi processone application runtimeone physical stateone MCP semantic interfacemultiple possible MCP clients

External agents therefore interact with the same application that already owns the sensors and physical hardware.

Exposing semantic tools instead of hardware functions The most important design decision is what the MCP server exposes. I could expose something like:

read_ens160()read_mpu6050()get_serial_message()

But that would make the AI client dependent on the hardware design. Instead, I prefer semantic tools. For example:

get_status()get_physical_context()get_presence()

The agent asks for information about the physical environment. The Raspberry Pi decides which hardware and services are required to answer the request. This makes the MCP interface independent from individual sensor implementations. In PHOS, the open-source project where I implemented this architecture, the MCP layer currently exposes semantic tools including:

phos_get_statusphos_get_physical_contextphos_get_presencephos_get_attentionphos_get_observed_expression

and actions such as:

phos_set_expressionphos_speakphos_sleepphos_wakephos_signal_user

These tools go through the application-service layer rather than directly manipulating hardware. The prefix is project-specific. The architectural pattern is not. A home-automation project could expose:

get_room_contextset_room_light

A greenhouse could expose:

get_greenhouse_statusstart_irrigation

A laboratory device might expose:

get_measurement_contextstart_experiment

MCP lets the physical application define capabilities in terms that make sense to an AI agent.

Building a Semantic Physical Context One particularly useful abstraction is a single tool that answers:

What does the Raspberry Pi know about the physical world right now?

For example:

{  "environment": {    "available": true,    "temperature_c": 24.3,    "humidity_percent": 51,    "pressure_hpa": 1013,    "condition": "comfortable"  },  "air_quality": {    "available": true,    "eco2_ppm": 540,    "tvoc_ppb": 61,    "condition": "good"  },  "motion": {    "moving": false  }}

The AI agent does not need to know:

which I2C bus is usedwhich ESP32 library reads the sensorhow the serial protocol workswhich Raspberry Pi process owns the connection

It receives a semantic description. This is where Raspberry Pi becomes more than a gateway. It becomes the semantic bridge between AI and the physical world.

Example — Asking an LLM About the Room Now we can combine the entire architecture.

Imagine the user asks:

How is the room?

The LLM agent decides it needs physical information. It calls:

get_physical_context()

through MCP. The request reaches the Raspberry Pi. The Raspberry Pi already has the latest sensor state received from the ESP32.

It can return something like:

{  "environment": {    "temperature_c": 27.8,    "humidity_percent": 63,    "condition": "warm"  },  "air_quality": {    "eco2_ppm": 620,    "condition": "good"  }}

The LLM can now reason:

27.8°C+63% humidity+good air quality

and answer:

The room is quite warm and slightly humid, but the current air quality looks good.

The complete flow is:

User ↓LLM Agent ↓MCP tool call ↓Raspberry Pi ↓semantic physical context ↓latest ESP32 sensor information ↓Raspberry Pi ↓MCP result ↓LLM reasoning ↓natural-language answer

Notice that the LLM never touched the sensor. It never opened /dev/ttyUSBx. It never read an ENS160 register. It never needed to know an MPU6050 existed. That complexity stays where it belongs: inside the physical application.

MCP also supports resources. The implementation I built currently exposes resources such as:

phos://statusphos://physical-contextphos://environmentphos://vision/snapshot

This means the Raspberry Pi can expose not only sensor state but also information generated by other physical subsystems. For example, the Raspberry Pi camera is owned by the normal vision pipeline. The MCP server can expose the latest available JPEG frame without opening a second camera instance. That path has been tested on real Raspberry Pi hardware: a camera JPEG was retrieved through MCP and opened on another computer.

This extends the same architecture:

Sensors ──┐Camera  ──┤Audio   ──┤         Raspberry Pi             ↓          semantic layer             ↓             MCP             ↓          LLM Agent

The Raspberry Pi remains responsible for acquiring physical information. MCP determines how an external AI client accesses it.

MCP Transport For the Raspberry Pi side, I use MCP Streamable HTTP. That matches the lifecycle of a physical device. The Pi starts and keeps running independently:

Raspberry Pi starts       ↓application starts       ↓MCP server starts       ↓agent connects       ↓agent disconnects       ↓Raspberry Pi keeps running

This is preferable to making the AI client responsible for starting the Raspberry Pi application. Some MCP clients instead expect a local stdio server. In that situation a small compatibility bridge can be used:

LLM Client    ↓ MCP stdioLocal Bridge    ↓ MCP Streamable HTTPRaspberry Pi

The bridge only changes the transport. It should not duplicate Raspberry Pi business logic. This also means different LLM clients can use the same Raspberry Pi MCP server.

Tool Discovery Another useful property of MCP is dynamic tool discovery. The client can request:

tools/list

and the Raspberry Pi returns the available capabilities. The flow becomes:

LLM Client   ↓ tools/listMCP Server on Raspberry Pi   ↓tool definitions   ↓LLM Client

That means the Raspberry Pi is the authoritative source of its capabilities. If I later add:

get_light_level

or:

set_led_signal

the MCP client can discover the new tool without having its own hardcoded copy of the hardware interface.

Why Raspberry Pi Is the Key Component The ESP32 is very good at:

reading sensorscontrolling LEDshandling deterministic hardware tasks

An LLM is very good at:

reasoninginterpreting contextplanninggenerating responses

Raspberry Pi sits exactly between those two worlds.

That gives us a clean division of responsibility:

ESP32=physical acquisitionRaspberry Pi=semantic integrationLLM Agent=reasoning

For me, that is the main lesson from this project.

PHOS — The Open-Source Implementation The system described here is currently implemented as part of PHOS, an open-source Raspberry Pi robotics project. It is simply the real implementation I used to test the architecture.

If you want to experiment with connecting an LLM agent to Raspberry Pi hardware, the repository provides one working architecture to study and extend.

[Read more](<javascript:void(0)>)

── more in #ai-agents 4 stories · sorted by recency
── more on @raspberry pi 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/connecting-raspberry…] indexed:0 read:10min 2026-11-16 · —