{"slug": "controlling-the-human-body-via-ems", "title": "Controlling the Human Body via EMS", "summary": "Researchers at MIT developed Human Operator, a wearable system that uses voice commands and first-person vision to control electrical muscle stimulation (EMS) for human augmentation. The system won MIT Hard Mode 2026 and maps AI-generated actions to relay-routed EMS, enabling tasks like finger movement and wrist stimulation.", "body_md": "Human Operator is a wearable human-augmentation system that maps voice + first-person vision input to relay-routed Electrical Muscle Stimulation (EMS) actions.\n\nThis project won MIT Hard Mode 2026 (Learn Track).\n\nLeft to right: AI stimulates wrist muscle to wave • AI stimulates fingers in sequence to play melody • AI stimulates fingers to form an OK sign\n\n`app.py`\n\n: Main runtime loop (camera + voice trigger + LLM + command dispatch)`utils/receiver.py`\n\n: Flask hardware gateway for timed relay/EMS command execution`manual_control_app.py`\n\n: PyQt GUI for manual calibration and direct stimulation testing`firmware/human_operator_ems/human_operator_ems.ino`\n\n: Arduino relay controller firmware`run_hardware.sh`\n\n: Recommended one-command launcher for real relay hardware mode\n\n- Peter He -\n[Portfolio](https://peterhci.com)|[GitHub](https://github.com/molegod)|[LinkedIn](https://www.linkedin.com/in/ph475/) - Valdemar Danry -\n[Portfolio](https://valdemardanry.com)|[GitHub](https://github.com/valleballe)|[LinkedIn](https://www.linkedin.com/in/valdemar-danry) - Daniel Kaijzer -\n[GitHub](https://github.com/danielkaijzer)|[LinkedIn](https://www.linkedin.com/in/danielkaijzer/) - Yutong Wu -\n[GitHub](https://github.com/ichbinHallie0426)|[LinkedIn](https://www.linkedin.com/in/yutong-wu-4b66661b5/) - Ashley Neall -\n[Portfolio](https://aneall.github.io/)|[GitHub](https://github.com/aneall)|[LinkedIn](https://www.linkedin.com/in/ashley-neall/) - Sean Hardesty Lewis -\n[Portfolio](https://seanhardestylewis.com/)|[GitHub](https://github.com/seanhlewis)|[LinkedIn](https://www.linkedin.com/in/seanhardestylewis/)\n\nThis repository controls physical electrical stimulation hardware.\n\n- Use only with appropriate supervision and informed consent.\n- Keep stimulation intensities conservative during calibration.\n- Ensure emergency stop access (software stop and physical disconnect).\n- Do not use on people with contraindications to EMS.\n\n- Python 3.10+\n- macOS/Linux shell environment\n- Arduino IDE (for flashing firmware)\n- Anthropic API key (\n`ANTHROPIC_API_KEY`\n\n) - Hardware:\n- Relay MCU (Arduino-compatible)\n- Relay board\n- EMS/stimulator device (optional in relay-only mode)\n- Camera for first-person capture\n\n- Clone and enter the project:\n\n```\ngit clone https://github.com/danielkaijzer/Human-Operator.git\ncd Human-Operator\n```\n\n- Create and activate a local virtual environment:\n\n```\npython3 -m venv .venv\nsource .venv/bin/activate\npip install -r requirements.txt\npip install pyserial requests\n```\n\n- Configure environment variables (LLM key):\n\n```\ncp .env_empty .env\n# then edit .env and set:\n# ANTHROPIC_API_KEY=your_key_here\n```\n\n- Flash relay firmware to your Arduino-compatible board:\n\n- Open\n`firmware/human_operator_ems/human_operator_ems.ino`\n\n- Select correct board/port in Arduino IDE\n- Upload at 115200 baud serial settings\n\n- Run full stack in relay-only hardware mode:\n\n```\n./run_hardware.sh\n```\n\nOptional custom relay port:\n\n```\nRELAY_PORT=/dev/cu.wchusbserial10 ./run_hardware.sh\n```\n\nWhat `run_hardware.sh`\n\ndoes:\n\n- Starts\n`utils/receiver.py`\n\nin`HARDWARE_MODE=relay`\n\n- Waits for\n`/health`\n\n- Verifies\n`relay_hardware_connected=true`\n\n- Launches\n`app.py`\n\nwith`RECEIVER_URL`\n\nset - Cleans up background receiver process on exit\n\nUse this for reliable relay switching and wiring validation:\n\n```\nHARDWARE_MODE=relay RELAY_PORT=/dev/cu.usbserial-210 python utils/receiver.py\n```\n\nCheck health from another terminal:\n\n```\ncurl -sS http://127.0.0.1:5001/health\n```\n\nExpected key fields:\n\n`\"hardware_mode\": \"relay\"`\n\n`\"relay_hardware_connected\": true`\n\nEnable full stim path explicitly:\n\n```\nHARDWARE_MODE=full STIM_PORT=/dev/cu.usbserial-XXX RELAY_PORT=/dev/cu.usbserial-YYY python utils/receiver.py\n```\n\nNotes:\n\n- Stim is intentionally disabled unless full mode is explicitly enabled.\n- If a device shows as\n`SIMULATED`\n\n, that serial port failed to open.\n\nStart the app (if not using `run_hardware.sh`\n\n):\n\n```\npython app.py\n```\n\nRuntime flow:\n\n- Wait for voice command trigger.\n- Capture latest camera frame.\n- Send prompt + image to Claude.\n- Transform action plan into timestamped relay/EMS payload.\n- POST payload to\n`utils/receiver.py`\n\nat`RECEIVER_URL`\n\n.\n\nRun:\n\n```\npython manual_control_app.py\n```\n\nUse it to:\n\n- Connect EMS and relay serial ports manually\n- Select relay targets (\n`wrist_left`\n\n,`wrist_right`\n\n,`thumb`\n\n,`index`\n\n,`middle`\n\n,`ring`\n\n,`pinky`\n\n,`x`\n\n) - Tune amplitude/frequency/pulse width\n- Validate response before autonomous runtime\n\nRelay selector commands accepted by receiver/firmware:\n\n`wrist_left`\n\n`wrist_right`\n\n`thumb`\n\n`index`\n\n`middle`\n\n`ring`\n\n`pinky`\n\n`x`\n\n(all off / isolate)\n\nFirmware pin mapping in `firmware/human_operator_ems/human_operator_ems.ino`\n\n:\n\n- D2 ->\n`wrist_left`\n\n- D4 ->\n`wrist_right`\n\n- D3 ->\n`thumb`\n\n- D5 ->\n`index`\n\n- D6 ->\n`middle`\n\n- D7 ->\n`ring`\n\n- D8 ->\n`pinky`\n\nFirmware serial debug commands:\n\n`test`\n\nto sweep all outputs`mode_low`\n\nfor active-low relay boards`mode_high`\n\nfor active-high relay boards`x`\n\nto set all outputs off\n\nReturns receiver status and hardware connection details.\n\nAccepts timestamp-keyed command dictionaries. Example:\n\n```\n{\n  \"0.0\": [\n    {\"type\": \"RELAY\", \"finger\": \"index\"},\n    {\"type\": \"EMS\", \"channel\": 1, \"amplitude\": 60, \"duration\": 1.0, \"frequency\": 100, \"pulse_width\": 1000}\n  ],\n  \"1.5\": [\n    {\"type\": \"RELAY\", \"finger\": \"x\"}\n  ]\n}\n```\n\nBall-triggered avoidance demo:\n\n```\npython utils/ball_demo.py\n```\n\n- Symptom:\n`\"relay\": \"SIMULATED\"`\n\nin`/execute`\n\nresponse or`/health`\n\n- Cause: relay serial port failed to open (busy/wrong port/power issue)\n- Fixes:\n- Close Arduino Serial Monitor and any other serial tool\n- Verify correct\n`RELAY_PORT`\n\n- Re-run in explicit relay mode:\n`HARDWARE_MODE=relay RELAY_PORT=/dev/cu.usbserial-210 python utils/receiver.py`\n\n- Run firmware\n`test`\n\ncommand over serial - Try\n`mode_low`\n\nand`mode_high`\n\n- Verify relay board power and common ground\n- Verify IN pin wiring vs firmware pin map\n\n- Confirm camera permissions in macOS privacy settings\n- Confirm no other app is locking the camera\n\n- Ensure\n`.env`\n\nexists and includes`ANTHROPIC_API_KEY`\n\n- Check network connectivity and Anthropic account access\n\n``` php\napp.py\n  -> captures camera frame and voice command\n  -> asks Claude for action plan\n  -> transforms plan into receiver payload\n  -> POST /execute to utils/receiver.py\n\nutils/receiver.py (Flask)\n  -> validates and executes timestamped sequence\n  -> sends relay selector command to Arduino firmware\n  -> sends optional EMS/GVS/ET command to stimulator\n\nfirmware/human_operator_ems/human_operator_ems.ino\n  -> maps symbolic relay targets to physical output pins\n```\n\nInspired by work from the [Human Computer Integration Lab](https://lab.plopes.org/) at UChicago:", "url": "https://wpnews.pro/news/controlling-the-human-body-via-ems", "canonical_source": "https://github.com/danielkaijzer/Human-Operator", "published_at": "2026-07-10 04:27:57+00:00", "updated_at": "2026-07-10 04:35:37.760507+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-products", "ai-research", "robotics"], "entities": ["MIT", "Human Operator", "Peter He", "Valdemar Danry", "Daniel Kaijzer", "Yutong Wu", "Ashley Neall", "Sean Hardesty Lewis"], "alternates": {"html": "https://wpnews.pro/news/controlling-the-human-body-via-ems", "markdown": "https://wpnews.pro/news/controlling-the-human-body-via-ems.md", "text": "https://wpnews.pro/news/controlling-the-human-body-via-ems.txt", "jsonld": "https://wpnews.pro/news/controlling-the-human-body-via-ems.jsonld"}}