# Connecting Raspberry Pi to LLM Agents Through MCP

> Source: <https://www.hackster.io/survivingwithandroid/connecting-raspberry-pi-to-llm-agents-through-mcp-b32390>
> Published: 2026-11-16 08:22:29+00:00

Large Language Models are very good at working with digital information. They can reason over documents, call APIs, analyze data and invoke software tools. But what happens when we want an LLM agent to know what is happening in the physical world?

Can it know the temperature of a room?

Can it detect whether someone is nearby?

Can it access air-quality information?

Can it trigger a physical response on a device?

This project started from that question.

The architecture I built uses a **Raspberry Pi as the connection point between an LLM agent and the physical environment**.

An ESP32 handles sensors and low-level hardware acquisition. The Raspberry Pi receives that information, interprets it and exposes semantic capabilities through the **Model Context Protocol — MCP**.

This video shows the final result:

The resulting architecture is:

```
Physical World      ↓   Sensors      ↓    ESP32      ↓ USBRaspberry Pi      ↓Semantic Services      ↓ MCP Server      ↓  LLM Agent
```

The goal is not to give an LLM direct access to hardware.

The goal is to let the Raspberry Pi expose meaningful information about the physical world in a form an AI agent can safely understand and use.

Components
For the implementation described here I use:

- Raspberry Pi (I have used Raspberry Pi 3 but you can use 4 or 5 too)
- ESP32 development board
- USB cable between ESP32 and Raspberry Pi
- BME280 or BMP280 environmental sensor
- AHT21 humidity/temperature sensor
- ENS160 air-quality sensor
- MPU6050 IMU
- optional WS2812B RGB LED ring
- Raspberry Pi camera

Examples of MCP-compatible clients can include desktop or development environments built around models such as Claude, ChatGPT or other LLM-based agents. The specific LLM is not important to the architecture.

Step 1 — Connecting an LLM Agent to the Physical World
The first design decision was deciding **where the intelligence should live**. I did not want the LLM agent to become the hardware controller.

For example, I did not want an agent to execute operations such as:

```
read GPIO 17read I2C register 0x21write raw LED valueopen serial port
```

Those are implementation details. Instead, I wanted the interaction to look more like this:

```
What is the temperature?How is the air quality?Is somebody present?What is happening in the room?
```

This immediately creates an architectural boundary. The AI agent handles:

```
reasoninginterpretationplanningdecision making
```

The Raspberry Pi handles:

```
device statesensor interpretationapplication logicpermissionshardware integration
```

MCP sits between them. A useful way to think about the architecture is:

```
LLM Agent    ↓semantic request    ↓Raspberry Pi    ↓physical system
```

instead of:

```
LLM Agent    ↓hardware driver
```

This distinction becomes increasingly important when AI agents move from software into robotics, IoT and other physical systems.

The next problem was hardware acquisition. A Raspberry Pi can communicate directly with I²C sensors, but once the number of peripherals starts growing, I prefer separating real-time hardware handling from the application runtime. For this project I use an **ESP32 as a dedicated controller**. It manages sensors including:

```
BME280 / BMP280AHT21ENS160MPU6050
```

and can also control devices such as a WS2812B RGB LED ring. The ESP32 is connected physically to the Raspberry Pi through **USB**. From the Raspberry Pi point of view, this becomes a serial communication channel. The hardware architecture is therefore:

```
BME280 ──┐AHT21  ──┤ENS160 ──┤MPU6050 ─┼── ESP32 ── USB Serial ── Raspberry PiLED Ring ─┘
```

This separation gives each device a clear responsibility.

ESP32 responsibilities
The ESP32 handles:

- environmental sensor acquisition
- air-quality measurements
- acceleration and angular velocity
- RGB LED execution
- controller heartbeat
- communication with Raspberry Pi

The Raspberry Pi therefore does not need to continuously manage every low-level sensor operation itself.

Sensor data
The environmental layer can provide values such as:

```
temperaturepressurehumidity
```

The ENS160 provides air-quality information including:

```
eCO2TVOCAQI
```

The MPU6050 provides:

```
accelerationangular velocity
```

These capabilities are part of the current controller-backed sensor architecture. But simply transmitting a number is not enough. Real sensors have states. They may be warming up. Data may be stale. Measurements may be technically available but not yet reliable.

For this reason the application distinguishes concepts such as:

```
measurement acquiredmeasurement freshmeasurement validmeasurement usable
```

That distinction is especially important when an LLM will eventually reason over the data. You do not want an AI agent confidently telling you that the air quality is poor because it received a value from a sensor that has not completed its initialization cycle.

Communication between ESP32 and Raspberry Pi
The ESP32 sends structured information to the Pi through the USB serial connection. In my implementation, communication uses a newline-delimited JSON protocol.

A simplified message could look like:

```
{  "type": "environment",  "temperature": 23.8,  "humidity": 47.2,  "pressure": 1014.1}
```

Another message could represent motion:

```
{  "type": "motion",  "acceleration": {    "x": 0.02,    "y": -0.01,    "z": 0.98  }}
```

The Raspberry Pi receives these messages and updates its internal view of the physical environment. The ESP32 knows how to read the hardware. The Raspberry Pi knows what the measurements mean to the application. That boundary is one of the most useful parts of the architecture.

Once the Raspberry Pi has access to sensor information, the next step is making that information available to an LLM agent. This is where MCP becomes useful. Instead of designing a custom API specifically for one AI model, the Raspberry Pi exposes an **MCP server**. The important part is that MCP is not another hardware runtime. It is an adapter over the Raspberry Pi application.

The architecture becomes:

```
LLM Agent      ↓     MCP      ↓Raspberry Pi MCP Server      ↓Application Services      ↓Semantic Physical Context      ↓ESP32 / Camera / Audio / Hardware
```

The MCP server runs inside the existing Raspberry Pi runtime instead of creating a second copy of the physical application. This means:

```
one Raspberry Pi processone application runtimeone physical stateone MCP semantic interfacemultiple possible MCP clients
```

External agents therefore interact with the same application that already owns the sensors and physical hardware.

Exposing semantic tools instead of hardware functions
The most important design decision is what the MCP server exposes. I could expose something like:

```
read_ens160()read_mpu6050()get_serial_message()
```

But that would make the AI client dependent on the hardware design. Instead, I prefer semantic tools. For example:

```
get_status()get_physical_context()get_presence()
```

The agent asks for information about the physical environment. The Raspberry Pi decides which hardware and services are required to answer the request. This makes the MCP interface independent from individual sensor implementations. In PHOS, the open-source project where I implemented this architecture, the MCP layer currently exposes semantic tools including:

```
phos_get_statusphos_get_physical_contextphos_get_presencephos_get_attentionphos_get_observed_expression
```

and actions such as:

```
phos_set_expressionphos_speakphos_sleepphos_wakephos_signal_user
```

These tools go through the application-service layer rather than directly manipulating hardware. *The prefix is project-specific*. The architectural pattern is not. A home-automation project could expose:

```
get_room_contextset_room_light
```

A greenhouse could expose:

```
get_greenhouse_statusstart_irrigation
```

A laboratory device might expose:

```
get_measurement_contextstart_experiment
```

MCP lets the physical application define capabilities in terms that make sense to an AI agent.

Building a Semantic Physical Context
One particularly useful abstraction is a single tool that answers:

What does the Raspberry Pi know about the physical world right now?

For example:

```
{  "environment": {    "available": true,    "temperature_c": 24.3,    "humidity_percent": 51,    "pressure_hpa": 1013,    "condition": "comfortable"  },  "air_quality": {    "available": true,    "eco2_ppm": 540,    "tvoc_ppb": 61,    "condition": "good"  },  "motion": {    "moving": false  }}
```

The AI agent does not need to know:

```
which I2C bus is usedwhich ESP32 library reads the sensorhow the serial protocol workswhich Raspberry Pi process owns the connection
```

It receives a semantic description. This is where Raspberry Pi becomes more than a gateway. It becomes the **semantic bridge between AI and the physical world**.

Example — Asking an LLM About the Room Now we can combine the entire architecture.

Imagine the user asks:

How is the room?

The LLM agent decides it needs physical information. It calls:

```
get_physical_context()
```

through MCP. The request reaches the Raspberry Pi. The Raspberry Pi already has the latest sensor state received from the ESP32.

It can return something like:

```
{  "environment": {    "temperature_c": 27.8,    "humidity_percent": 63,    "condition": "warm"  },  "air_quality": {    "eco2_ppm": 620,    "condition": "good"  }}
```

The LLM can now reason:

```
27.8°C+63% humidity+good air quality
```

and answer:

The room is quite warm and slightly humid, but the current air quality looks good.

The complete flow is:

```
User ↓LLM Agent ↓MCP tool call ↓Raspberry Pi ↓semantic physical context ↓latest ESP32 sensor information ↓Raspberry Pi ↓MCP result ↓LLM reasoning ↓natural-language answer
```

Notice that the LLM never touched the sensor. It never opened `/dev/ttyUSBx`. It never read an ENS160 register. It never needed to know an MPU6050 existed. That complexity stays where it belongs: inside the physical application.

MCP also supports resources. The implementation I built currently exposes resources such as:

```
phos://statusphos://physical-contextphos://environmentphos://vision/snapshot
```

This means the Raspberry Pi can expose not only sensor state but also information generated by other physical subsystems. For example, the Raspberry Pi camera is owned by the normal vision pipeline. The MCP server can expose the latest available JPEG frame without opening a second camera instance. That path has been tested on real Raspberry Pi hardware: a camera JPEG was retrieved through MCP and opened on another computer.

This extends the same architecture:

```
Sensors ──┐Camera  ──┤Audio   ──┤         Raspberry Pi             ↓          semantic layer             ↓             MCP             ↓          LLM Agent
```

The Raspberry Pi remains responsible for acquiring physical information. MCP determines how an external AI client accesses it.

MCP Transport
For the Raspberry Pi side, I use **MCP Streamable HTTP**. That matches the lifecycle of a physical device. The Pi starts and keeps running independently:

```
Raspberry Pi starts       ↓application starts       ↓MCP server starts       ↓agent connects       ↓agent disconnects       ↓Raspberry Pi keeps running
```

This is preferable to making the AI client responsible for starting the Raspberry Pi application. Some MCP clients instead expect a local `stdio` server. In that situation a small compatibility bridge can be used:

```
LLM Client    ↓ MCP stdioLocal Bridge    ↓ MCP Streamable HTTPRaspberry Pi
```

The bridge only changes the transport. It should not duplicate Raspberry Pi business logic. This also means different LLM clients can use the same Raspberry Pi MCP server.

Tool Discovery
Another useful property of MCP is dynamic tool discovery. The client can request:

```
tools/list
```

and the Raspberry Pi returns the available capabilities. The flow becomes:

```
LLM Client   ↓ tools/listMCP Server on Raspberry Pi   ↓tool definitions   ↓LLM Client
```

That means the Raspberry Pi is the authoritative source of its capabilities. If I later add:

```
get_light_level
```

or:

```
set_led_signal
```

the MCP client can discover the new tool without having its own hardcoded copy of the hardware interface.

Why Raspberry Pi Is the Key Component
The ESP32 is very good at:

```
reading sensorscontrolling LEDshandling deterministic hardware tasks
```

An LLM is very good at:

```
reasoninginterpreting contextplanninggenerating responses
```

Raspberry Pi sits exactly between those two worlds.

That gives us a clean division of responsibility:

```
ESP32=physical acquisitionRaspberry Pi=semantic integrationLLM Agent=reasoning
```

For me, that is the main lesson from this project.

PHOS — The Open-Source Implementation
The system described here is currently implemented as part of **PHOS**, an open-source Raspberry Pi robotics project. It is simply the real implementation I used to test the architecture.

If you want to experiment with connecting an LLM agent to Raspberry Pi hardware, the repository provides one working architecture to study and extend.

[Read more](<javascript:void(0)>)
