# Controlling the Human Body via EMS

> Source: <https://github.com/danielkaijzer/Human-Operator>
> Published: 2026-07-10 04:27:57+00:00

Human Operator is a wearable human-augmentation system that maps voice + first-person vision input to relay-routed Electrical Muscle Stimulation (EMS) actions.

This project won MIT Hard Mode 2026 (Learn Track).

Left to right: AI stimulates wrist muscle to wave • AI stimulates fingers in sequence to play melody • AI stimulates fingers to form an OK sign

`app.py`

: Main runtime loop (camera + voice trigger + LLM + command dispatch)`utils/receiver.py`

: Flask hardware gateway for timed relay/EMS command execution`manual_control_app.py`

: PyQt GUI for manual calibration and direct stimulation testing`firmware/human_operator_ems/human_operator_ems.ino`

: Arduino relay controller firmware`run_hardware.sh`

: Recommended one-command launcher for real relay hardware mode

- Peter He -
[Portfolio](https://peterhci.com)|[GitHub](https://github.com/molegod)|[LinkedIn](https://www.linkedin.com/in/ph475/) - Valdemar Danry -
[Portfolio](https://valdemardanry.com)|[GitHub](https://github.com/valleballe)|[LinkedIn](https://www.linkedin.com/in/valdemar-danry) - Daniel Kaijzer -
[GitHub](https://github.com/danielkaijzer)|[LinkedIn](https://www.linkedin.com/in/danielkaijzer/) - Yutong Wu -
[GitHub](https://github.com/ichbinHallie0426)|[LinkedIn](https://www.linkedin.com/in/yutong-wu-4b66661b5/) - Ashley Neall -
[Portfolio](https://aneall.github.io/)|[GitHub](https://github.com/aneall)|[LinkedIn](https://www.linkedin.com/in/ashley-neall/) - Sean Hardesty Lewis -
[Portfolio](https://seanhardestylewis.com/)|[GitHub](https://github.com/seanhlewis)|[LinkedIn](https://www.linkedin.com/in/seanhardestylewis/)

This repository controls physical electrical stimulation hardware.

- Use only with appropriate supervision and informed consent.
- Keep stimulation intensities conservative during calibration.
- Ensure emergency stop access (software stop and physical disconnect).
- Do not use on people with contraindications to EMS.

- Python 3.10+
- macOS/Linux shell environment
- Arduino IDE (for flashing firmware)
- Anthropic API key (
`ANTHROPIC_API_KEY`

) - Hardware:
- Relay MCU (Arduino-compatible)
- Relay board
- EMS/stimulator device (optional in relay-only mode)
- Camera for first-person capture

- Clone and enter the project:

```
git clone https://github.com/danielkaijzer/Human-Operator.git
cd Human-Operator
```

- Create and activate a local virtual environment:

```
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install pyserial requests
```

- Configure environment variables (LLM key):

```
cp .env_empty .env
# then edit .env and set:
# ANTHROPIC_API_KEY=your_key_here
```

- Flash relay firmware to your Arduino-compatible board:

- Open
`firmware/human_operator_ems/human_operator_ems.ino`

- Select correct board/port in Arduino IDE
- Upload at 115200 baud serial settings

- Run full stack in relay-only hardware mode:

```
./run_hardware.sh
```

Optional custom relay port:

```
RELAY_PORT=/dev/cu.wchusbserial10 ./run_hardware.sh
```

What `run_hardware.sh`

does:

- Starts
`utils/receiver.py`

in`HARDWARE_MODE=relay`

- Waits for
`/health`

- Verifies
`relay_hardware_connected=true`

- Launches
`app.py`

with`RECEIVER_URL`

set - Cleans up background receiver process on exit

Use this for reliable relay switching and wiring validation:

```
HARDWARE_MODE=relay RELAY_PORT=/dev/cu.usbserial-210 python utils/receiver.py
```

Check health from another terminal:

```
curl -sS http://127.0.0.1:5001/health
```

Expected key fields:

`"hardware_mode": "relay"`

`"relay_hardware_connected": true`

Enable full stim path explicitly:

```
HARDWARE_MODE=full STIM_PORT=/dev/cu.usbserial-XXX RELAY_PORT=/dev/cu.usbserial-YYY python utils/receiver.py
```

Notes:

- Stim is intentionally disabled unless full mode is explicitly enabled.
- If a device shows as
`SIMULATED`

, that serial port failed to open.

Start the app (if not using `run_hardware.sh`

):

```
python app.py
```

Runtime flow:

- Wait for voice command trigger.
- Capture latest camera frame.
- Send prompt + image to Claude.
- Transform action plan into timestamped relay/EMS payload.
- POST payload to
`utils/receiver.py`

at`RECEIVER_URL`

.

Run:

```
python manual_control_app.py
```

Use it to:

- Connect EMS and relay serial ports manually
- Select relay targets (
`wrist_left`

,`wrist_right`

,`thumb`

,`index`

,`middle`

,`ring`

,`pinky`

,`x`

) - Tune amplitude/frequency/pulse width
- Validate response before autonomous runtime

Relay selector commands accepted by receiver/firmware:

`wrist_left`

`wrist_right`

`thumb`

`index`

`middle`

`ring`

`pinky`

`x`

(all off / isolate)

Firmware pin mapping in `firmware/human_operator_ems/human_operator_ems.ino`

:

- D2 ->
`wrist_left`

- D4 ->
`wrist_right`

- D3 ->
`thumb`

- D5 ->
`index`

- D6 ->
`middle`

- D7 ->
`ring`

- D8 ->
`pinky`

Firmware serial debug commands:

`test`

to sweep all outputs`mode_low`

for active-low relay boards`mode_high`

for active-high relay boards`x`

to set all outputs off

Returns receiver status and hardware connection details.

Accepts timestamp-keyed command dictionaries. Example:

```
{
  "0.0": [
    {"type": "RELAY", "finger": "index"},
    {"type": "EMS", "channel": 1, "amplitude": 60, "duration": 1.0, "frequency": 100, "pulse_width": 1000}
  ],
  "1.5": [
    {"type": "RELAY", "finger": "x"}
  ]
}
```

Ball-triggered avoidance demo:

```
python utils/ball_demo.py
```

- Symptom:
`"relay": "SIMULATED"`

in`/execute`

response or`/health`

- Cause: relay serial port failed to open (busy/wrong port/power issue)
- Fixes:
- Close Arduino Serial Monitor and any other serial tool
- Verify correct
`RELAY_PORT`

- Re-run in explicit relay mode:
`HARDWARE_MODE=relay RELAY_PORT=/dev/cu.usbserial-210 python utils/receiver.py`

- Run firmware
`test`

command over serial - Try
`mode_low`

and`mode_high`

- Verify relay board power and common ground
- Verify IN pin wiring vs firmware pin map

- Confirm camera permissions in macOS privacy settings
- Confirm no other app is locking the camera

- Ensure
`.env`

exists and includes`ANTHROPIC_API_KEY`

- Check network connectivity and Anthropic account access

``` php
app.py
  -> captures camera frame and voice command
  -> asks Claude for action plan
  -> transforms plan into receiver payload
  -> POST /execute to utils/receiver.py

utils/receiver.py (Flask)
  -> validates and executes timestamped sequence
  -> sends relay selector command to Arduino firmware
  -> sends optional EMS/GVS/ET command to stimulator

firmware/human_operator_ems/human_operator_ems.ino
  -> maps symbolic relay targets to physical output pins
```

Inspired by work from the [Human Computer Integration Lab](https://lab.plopes.org/) at UChicago: