{"slug": "know-the-best-move-when-playing-chess-with-meta-glasses", "title": "Know the best move when playing chess with Meta glasses", "summary": "A project called chesshacker lets a person wearing Meta glasses play over-the-board chess by video-calling a Mac over WhatsApp and having the Mac read the board and speak Stockfish's best move back into the call. The vision model only reports the squares it sees, code matches that to a legal move, and Stockfish — never the language model — picks the reply; the default vision model gpt-6.1-sol read 3 of 3 test moves at 1.4¢ and 4–4.6 seconds per frame, versus 3 of 3 for claude-opus-5-5 at 4¢ and 3.6–6.3 seconds. The system was tested end to end on glasses recordings but not yet on a live WhatsApp call, and spending pauses at $10 by default via --max-cost.", "body_md": "## Overview\n\nchesshacker lets the person wearing Meta glasses play over-the-board chess with the best move in their ear. The glasses video-call a Mac on WhatsApp and share their view. The Mac works out when the opponent has moved, asks Stockfish for the reply, and speaks it back into the call.\n\nTwo rules keep it honest:\n\n- The vision model only reports what it **sees** . Code matches that to a legal move.\n- Stockfish picks the move, never the language model.\n\nTested end to end on glasses recordings. Not yet tried on a live WhatsApp call.\n\n## Setup\n\nYou need a Mac with Node.js 22 or newer (`brew install node`).\n\n**Clone this project** Make this project your own and edit it easily with Reverse.\n\n[Clone](https://www.reverse.dev/signup?from=chess)\n\nSetup downloads Stockfish, asks for your keys, and checks everything. You need an OpenAI API key; a Jev key from TypeSafe is optional and makes games cheaper. Keys are kept in your login Keychain.\n\nAlso install WhatsApp from the Mac App Store and BlackHole 2ch (`brew install blackhole-2ch`), and give your terminal app Screen Recording permission. Then check a real call once with `npx chesshacker test-call`.\n\n## Playing a game\n\n1. From the glasses, video-call this Mac’s WhatsApp and answer it on the Mac.\n2. Double-tap the glasses’ capture button to share their view. In the call’s ⋯ menu, set the microphone to BlackHole 2ch.\n3. Start chesshacker with your color:\n\n```\nnpx chesshacker run --color white\nnpx chesshacker run --color black --fen \"<position>\"\n```\n\nLook at the board. You’ll hear “Ready”, then the best move after each of your opponent’s moves. The command center opens in your browser at 127.0.0.1:8000, with the live frame, the board and its arrow, and controls to pause, undo, enter a move, or say it again.\n\nSpending pauses at $10 by default (`--max-cost`).\n\n## How a move is read\n\nAbout once a second, a frame gets two looks:\n\n1. **A quick check.** Jev skips it if the board isn’t readable (under 0.8), a hand is over it (over 0.5), or nothing changed (under 0.1).\n2. **A full read.** The vision model lists the squares that look different from the last confirmed frame.\n\nCode compares that with the board after every legal move, and after every move plus a reply. A move is committed once it’s the best fit on 2 frames in a row, by 3 or more squares in total.\n\n| Vision model | Moves | Per frame | \n|---|---|---|\n| `gpt-6.1-sol` (default) | 3 of 3 | 1.4¢ · 4–4.6 s | \n| `claude-opus-5-5` | 3 of 3 | 4¢ · 3.6–6.3 s | \n\nAbout 10 seconds pass from a move to the spoken reply, mostly the two model reads.\n\n## When something’s wrong\n\n### Not sure what’s missing\n\nRun `npx chesshacker doctor`. It checks Stockfish, your keys, WhatsApp, BlackHole and Screen Recording, and prints the exact fix for each problem.\n\n### It watches the wrong window\n\n`npx chesshacker windows` lists WhatsApp’s windows. Pick one with `--window`.\n\n### “Lost track of the board”\n\nFix it from the command center: undo, enter the move yourself, or start a new game from a FEN.\n\n### No speech in the glasses\n\nSet the call’s microphone to BlackHole 2ch in WhatsApp’s ⋯ menu. `test-call` checks this for you.", "url": "https://wpnews.pro/news/know-the-best-move-when-playing-chess-with-meta-glasses", "canonical_source": "https://www.reverse.dev/pjchess", "published_at": "2026-11-02 18:38:00+00:00", "updated_at": "2026-11-02 18:51:48.038226+00:00", "lang": "en", "topics": ["computer-vision", "ai-tools", "large-language-models", "ai-agents"], "entities": ["Meta", "WhatsApp", "Stockfish", "OpenAI", "TypeSafe", "Jev", "BlackHole 2ch", "chesshacker"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/know-the-best-move-when-playing-chess-with-meta-glasses", "markdown": "https://wpnews.pro/news/know-the-best-move-when-playing-chess-with-meta-glasses.md", "text": "https://wpnews.pro/news/know-the-best-move-when-playing-chess-with-meta-glasses.txt", "jsonld": "https://wpnews.pro/news/know-the-best-move-when-playing-chess-with-meta-glasses.jsonld"}}