Controlling the Human Body via EMS

Human Operator is a wearable human-augmentation system that maps voice + first-person vision input to relay-routed Electrical Muscle Stimulation (EMS) actions.

This project won MIT Hard Mode 2026 (Learn Track).

Left to right: AI stimulates wrist muscle to wave • AI stimulates fingers in sequence to play melody • AI stimulates fingers to form an OK sign

app.py

: Main runtime loop (camera + voice trigger + LLM + command dispatch)utils/receiver.py

: Flask hardware gateway for timed relay/EMS command executionmanual_control_app.py

: PyQt GUI for manual calibration and direct stimulation testingfirmware/human_operator_ems/human_operator_ems.ino

: Arduino relay controller firmwarerun_hardware.sh

: Recommended one-command launcher for real relay hardware mode

• Peter He - Portfolio|GitHub|LinkedIn - Valdemar Danry - Portfolio|GitHub|LinkedIn - Daniel Kaijzer - GitHub|LinkedIn - Yutong Wu - GitHub|LinkedIn - Ashley Neall - Portfolio|GitHub|LinkedIn - Sean Hardesty Lewis - Portfolio|GitHub|LinkedIn

This repository controls physical electrical stimulation hardware.

• Use only with appropriate supervision and informed consent.

• Keep stimulation intensities conservative during calibration.

• Ensure emergency stop access (software stop and physical disconnect).

• Do not use on people with contraindications to EMS.

• Python 3.10+

• macOS/Linux shell environment

• Arduino IDE (for flashing firmware)

• Anthropic API key ( ANTHROPIC_API_KEY

) - Hardware:

• Relay MCU (Arduino-compatible)

• Relay board

• EMS/stimulator device (optional in relay-only mode)

• Camera for first-person capture

• Clone and enter the project:

git clone https://github.com/danielkaijzer/Human-Operator.git
cd Human-Operator

• Create and activate a local virtual environment:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install pyserial requests

• Configure environment variables (LLM key):

cp .env_empty .env

• Flash relay firmware to your Arduino-compatible board:

• Open firmware/human_operator_ems/human_operator_ems.ino

• Select correct board/port in Arduino IDE

• Upload at 115200 baud serial settings

• Run full stack in relay-only hardware mode:

./run_hardware.sh

Optional custom relay port:

RELAY_PORT=/dev/cu.wchusbserial10 ./run_hardware.sh

What run_hardware.sh

does:

• Starts utils/receiver.py

inHARDWARE_MODE=relay

• Waits for /health

• Verifies relay_hardware_connected=true

• Launches app.py

withRECEIVER_URL

set - Cleans up background receiver process on exit

Use this for reliable relay switching and wiring validation:

HARDWARE_MODE=relay RELAY_PORT=/dev/cu.usbserial-210 python utils/receiver.py

Check health from another terminal:

curl -sS http://127.0.0.1:5001/health

Expected key fields:

"hardware_mode": "relay"

"relay_hardware_connected": true

Enable full stim path explicitly:

HARDWARE_MODE=full STIM_PORT=/dev/cu.usbserial-XXX RELAY_PORT=/dev/cu.usbserial-YYY python utils/receiver.py

Notes:

• Stim is intentionally disabled unless full mode is explicitly enabled.
• If a device shows as SIMULATED

, that serial port failed to open.

Start the app (if not using run_hardware.sh

):

python app.py

Runtime flow:

• Wait for voice command trigger.
• Capture latest camera frame.
• Send prompt + image to Claude.
• Transform action plan into timestamped relay/EMS payload.
• POST payload to utils/receiver.py

atRECEIVER_URL

.

Run:

python manual_control_app.py

Use it to:

• Connect EMS and relay serial ports manually
• Select relay targets ( wrist_left

,wrist_right

,thumb

,index

,middle

,ring

,pinky

,x

) - Tune amplitude/frequency/pulse width

• Validate response before autonomous runtime

Relay selector commands accepted by receiver/firmware:

wrist_left

wrist_right

thumb

index

middle

ring

pinky

x

(all off / isolate)

Firmware pin mapping in firmware/human_operator_ems/human_operator_ems.ino

:

• D2 -> wrist_left

• D4 -> wrist_right

• D3 -> thumb

• D5 -> index

• D6 -> middle

• D7 -> ring

• D8 -> pinky

Firmware serial debug commands:

test

to sweep all outputsmode_low

for active-low relay boardsmode_high

for active-high relay boardsx

to set all outputs off

Returns receiver status and hardware connection details.

Accepts timestamp-keyed command dictionaries. Example:

{
  "0.0": [
    {"type": "RELAY", "finger": "index"},
    {"type": "EMS", "channel": 1, "amplitude": 60, "duration": 1.0, "frequency": 100, "pulse_width": 1000}
  ],
  "1.5": [
    {"type": "RELAY", "finger": "x"}
  ]
}

Ball-triggered avoidance demo:

python utils/ball_demo.py

• Symptom: "relay": "SIMULATED"

in/execute

response or/health

• Cause: relay serial port failed to open (busy/wrong port/power issue)

• Fixes:

• Close Arduino Serial Monitor and any other serial tool

• Verify correct RELAY_PORT

• Re-run in explicit relay mode: HARDWARE_MODE=relay RELAY_PORT=/dev/cu.usbserial-210 python utils/receiver.py

• Run firmware test

command over serial - Try mode_low

andmode_high

• Verify relay board power and common ground

• Verify IN pin wiring vs firmware pin map

• Confirm camera permissions in macOS privacy settings

• Confirm no other app is locking the camera

• Ensure .env

exists and includesANTHROPIC_API_KEY

• Check network connectivity and Anthropic account access

app.py
  -> captures camera frame and voice command
  -> asks Claude for action plan
  -> transforms plan into receiver payload
  -> POST /execute to utils/receiver.py

utils/receiver.py (Flask)
  -> validates and executes timestamped sequence
  -> sends relay selector command to Arduino firmware
  -> sends optional EMS/GVS/ET command to stimulator

firmware/human_operator_ems/human_operator_ems.ino
  -> maps symbolic relay targets to physical output pins

Inspired by work from the Human Computer Integration Lab at UChicago:

source & further reading

github.com — original article