{"slug": "building-a-physical-ai-that-plays-connect-4", "title": "Building a Physical AI that Plays Connect 4", "summary": "Roboflow published a tutorial by Aarnav Shah detailing a DIY Connect 4 robot that uses a HiWonder MaxArm, RF-DETR object detection, a negamax engine, and Gemini 3.6 Flash with ElevenLabs to play and trash-talk. The system detects board states in 175 ms per frame, evaluates positions in 12 ms, and achieved an F1 score of 0.96 at a 0.40 confidence threshold on its test split.", "body_md": "*Teaching a robotic arm to play Connect 4 and trash-talk human players in real time. The system combines local RF-DETR object detection for board tracking, a bitboard negamax engine that searches eight moves ahead in 12 milliseconds, a HiWonder MaxArm suction feeder setup, and Gemini Flash driving ElevenLabs voice roasts.*\n\nMy brother and I played Connect 4 constantly growing up, and almost every game devolved into a heated argument. So before leaving for university, I taught a physical robot arm to take my place and insult his moves out loud while beating him.\n\nThe system runs local [ computer vision](https://blog.roboflow.com/what-is-computer-vision/) at 175 milliseconds per frame, evaluates board positions in 12 milliseconds, and drops pieces automatically.\n\nThe [ large language model](https://blog.roboflow.com/prompting-tips-for-large-language-models-with-vision/) never selects game moves. A classic minimax algorithm handles all tactical decision-making. The language model acts only as the robot's vocal mouth.\n\nFollow along with the complete source code in our [ Connect 4 Robot GitHub repository](https://github.com/aarnavshah12/connect4-robot?ref=blog.roboflow.com).\n\n## System architecture\n\nWe divided the robot pipeline into four parts.\n\n- A custom\nsmall model running locally processes camera frames. A homography maps piece locations directly to a 7x6 grid and waits for five matching frames in a row so it never misreads a move.__RF-DETR__ - Next, a\nconverts the board state into bitboards, using fast binary math to evaluate positions at depth 8. In simple terms, it searches eight moves ahead in just 12 milliseconds. An automated test suite forces the engine through 20 random games to verify it never misses a forced win or a crucial block.__negamax game engine__ - Then, the\nand a suction nozzle picks pieces from a stack and drops them into designated columns.__HiWonder MaxArm__ - Lastly,\nwrites contextual roasts within a 4.5 second budget. It sends text to an ElevenLabs voice generator for playback.__Gemini 3.6 Flash__\n\n### Hardware list\n\nBuilding a physical board game requires some hardware.\n\n: a $200 desktop robot arm driven by an ESP32 controller and equipped with a suction nozzle.__HiWonder MaxArm__: a standard plastic grid containing standard red and yellow pieces.__Connect 4 board__- Some cardboard (what I used) or a 3D printer for clean, detachable parts\n\n## Training object detection with empty cell awareness\n\nWe trained our model (found from [ Roboflow Universe](https://universe.roboflow.com/projects-zidii/connect4?ref=blog.roboflow.com)) in the four classes: Red Piece, Yellow Piece, Board, and No Piece.\n\nDetecting empty cells directly sounds redundant. But it provided a health check for the camera pipeline.\n\nOn our test split, the model achieved an [ F1 score](https://blog.roboflow.com/f1-score/) of 0.96 at a 0.40\n\n[. The Red class registered perfect](https://blog.roboflow.com/object-detection-metrics/)\n\n__confidence threshold__[at 1.0.](https://blog.roboflow.com/precision-and-recall/)\n\n__precision__If you deploy local inference using our [ inference package](https://github.com/roboflow/inference?ref=blog.roboflow.com), use the project ID format\n\n`connect4-kewhf/1`\n\nrather than the raw workspace checkpoint path to avoid invalid model errors.## Building the cardboard top guide\n\nThe MaxArm relies on a vacuum suction nozzle, which grabs Connect 4 pieces flat from the feeder stack. Dropping a piece into the board requires vertical orientation. Physical robot arms also suffer from some mechanical backlash, making exact millimeter drops into narrow plastic slots unreliable.\n\nSo I built a custom cardboard funnel apparatus mounted directly across the top frame of the board. This can also be 3d printed if you have the resources:\n\nEach of the seven entry channels features a sloped interior wall designed to reorient the piece.\n\nWhen the suction cup releases a horizontal chip above the channel, the piece hits the slanted cardboard ramp and flips vertically as it slides into the column slot. This physical funnel fixes the orientation issue and increases drop tolerance, guiding pieces cleanly into the board even if the arm shifts off-target by a few millimeters.\n\n## Repository layout and setup\n\nSetting up the project requires a few shell commands.\n\n```\ngit clone https://github.com/aarnavshah12/connect4-robot.git\ncd connect4-robot\nuv venv .venv --python 3.12\nuv pip install --python .venv/bin/python -r requirements.txt\ncp config.example.yaml config.yaml\n```\n\nMake sure to add all of the API keys needed in `config.yaml`\n\n. If they’re missing, the system will use default text lines as a substitute for Gemini, and the macOS system voice substitutes for ElevenLabs. The core engine and vision system run entirely locally.\n\n## Calibration and setup\n\nIf you build your own version, secure your hardware before recording any points. Screw down the robot arm and anchor your piece feeder directly to the table. A shift of even two millimeters breaks suction alignment and invalidates your stored coordinates.\n\nAccurate coordinate mapping requires measuring things yourself instead of using online measurements. When setting up our red piece feeder, assuming a standard 5 millimeter piece thickness caused alternating grab failures for me. The arm grabbed the first and third pieces but missed the second and fourth because the error in the numbers made the arm drift past the suction cup tolerance.\n\nReading back actual Z-axis coordinates from successful grabs (96 millimeters, 90 millimeters, and 83 millimeters) revealed a true piece thickness of 6.5 millimeters. I suggest recording every pick height manually during calibration, jotting down your exact coordinates, and testing your stack count repeatedly before running a full game.\n\n## Build your own\n\n- Photograph your board, label red pieces, yellow pieces, board boundaries, and empty cells in the Roboflow UI, then train an RF-DETR model.\n- Input all your API keys in\n`config.yaml`\n\n- Clone the repository and install dependencies using uv.\n- Flash MicroPython firmware to your robot arm and teach the feeder pick locations.\n- Construct a cardboard top guide for column slots to maximize drop tolerances.\n- Run calibration to align camera homography, verify board coordinates using debug tools, and start the game engine.\n\n**Further reading**\n\n**Cite this Post**\n\nUse the following entry to cite this post in your research:\n\n[Aarnav Shah](/author/aarnavshah/). (Aug 21, 2026).\nBuilding a Physical AI that Plays Connect 4. Roboflow Blog: https://blog.roboflow.com/building-a-physical-ai-that-plays-connect-4/", "url": "https://wpnews.pro/news/building-a-physical-ai-that-plays-connect-4", "canonical_source": "https://blog.roboflow.com/building-a-physical-ai-that-plays-connect-4/", "published_at": "2026-08-21 19:11:46+00:00", "updated_at": "2026-08-21 19:43:42.304138+00:00", "lang": "en", "topics": ["robotics", "computer-vision", "artificial-intelligence", "large-language-models", "ai-products"], "entities": ["Roboflow", "Aarnav Shah", "HiWonder MaxArm", "RF-DETR", "Gemini 3.6 Flash", "ElevenLabs", "Connect 4", "Roboflow Universe"], "alternates": {"html": "https://wpnews.pro/news/building-a-physical-ai-that-plays-connect-4", "markdown": "https://wpnews.pro/news/building-a-physical-ai-that-plays-connect-4.md", "text": "https://wpnews.pro/news/building-a-physical-ai-that-plays-connect-4.txt", "jsonld": "https://wpnews.pro/news/building-a-physical-ai-that-plays-connect-4.jsonld"}}