Show HN: Pigame – pi+LLM plays black-box browser games via observe/move tools

Developer ifoxhz released Pigame, an open-source game agent that combines Pi.dev's agent platform with an LLM such as DeepSeek to play black-box browser games through observe and move tools, with the first demo being Neon Snake. The project, codenamed GameMind, provides scaffolding where the LLM acts as the brain and the tools serve as eyes, hands, and feet, and it includes a patched version of the snake game that pauses between moves to prevent stale decisions. Pigame is available on GitHub and can be installed via Pi CLI, with smoke tests for CI and support for visible Chromium via WSLg.

English | 中文 /ifoxhz/pigame/blob/main/README.zh-CN.md General game agent on Pi https://pi.dev + LLM e.g. DeepSeek . Design codename: GameMind . First demo game: Neon Snake . Remote: git@github.com:ifoxhz/pigame.git Welcome.Contributions of any kind and any ideas are welcome — let’s make the Pi agent more fun. Pi + LLM = brain strategy / when to stop pigame = scaffolding only tools = eyes / hands / feet | Layer | Role | |---|---| Pi + LLM | Decide what to do; judge when the goal is done | game observe | Eyes — see the screen → AgentState | game move / game wait | Hands & feet — act / wait | npm run smoke: / /game eat | Test scaffolding only — hardcoded loops for CI; not the agent path | There is no game eat food / game survive tool. The model loops observe → move itself. | Tool | Notes | |---|---| Node.js | ≥ 20 recommended --experimental-strip-types | npm | Comes with Node | Pi CLI | Required for the agent path pi on PATH | LLM in Pi | e.g. DeepSeek configured in your Pi settings | WSLg / display | Only for visible Chromium HEADED=1 | git clone git@github.com:ifoxhz/pigame.git cd pigame or your local folder name npm install --prefix pigame npx --prefix pigame playwright install chromium Default game no extra clone : pigame/games/snake.html — a vendored Neon Snake with agent mode see below . Anyone who clones this repo gets the patched game automatically; Adapter loads it on /game connect . Only if you want the original continuous-loop game for comparison: git clone git@github.com:Digiman/mini-game-studio-ai-gen.git HTTPS: https://github.com/Digiman/mini-game-studio-ai-gen.git SNAKE HTML=./mini-game-studio-ai-gen/games/snake.html HEADED=1 pi -e ./.pi/extensions/pigame.ts Live site reference : https://digiman.github.io/mini-game-studio-ai-gen/ https://digiman.github.io/mini-game-studio-ai-gen/ No separate compile step for smokes: Node strip-types runs TypeScript directly. Root package.json only proxies scripts into pigame/ . git clone git@github.com:ifoxhz/pigame.git cd pigame or your local folder name npm install --prefix pigame npx --prefix pigame playwright install chromium cd /path/to/pigame HEADED=1 pi -e ./.pi/extensions/pigame.ts In the Pi TUI in order : /game connect /skill:eat-food Or paste this prompt instead of the skill: Use only game observe, game move, and optionally game wait. Prefer groundTruth grid coords. Eat one food: observe → move → repeat. When score increases, stop and report before/after score. Do not reverse 180°. Watch the Playwright Chromium window snake freezes between moves . When done: /game disconnect After code changes in a live session: /reload , then /game disconnect → /game connect again. Adapter + perception + I/O only no Pi / no LLM npm run smoke:eat npm run smoke:eat:headed npm run pack:pi Produces: dist/pi-pigame/ — installable directory dist/pigame-0.1.0.tgz — npm tarball Others install into Pi with: pi install /path/to/pigame/dist/pi-pigame or pi install /path/to/pigame/dist/pigame-0.1.0.tgz after you publish to npm: npm publish ./dist/pigame-0.1.0.tgz pi install npm:pigame From a git clone without packing , they can also: pi install ./pigame Then restart Pi or /reload . First time: npx playwright install chromium from the installed package dir if needed . Problem: Upstream Snake ticks on its own. While the LLM is in a long observe → think → move chain, the snake has already moved, so decisions are stale and play feels clumsy. What we ship: pigame/games/snake.html see pigame/games/README.md /ifoxhz/pigame/blob/main/pigame/games/README.md | Behavior | Meaning | |---|---| Auto gameLoop | Paused after connect setAgentMode true | game observe | World stays frozen no time pass | game move | setDir + exactly 1 grid step MOVE STEPS to change | No special startup flag is required for agent mode — connect uses the vendored file by default. | Variable | Default | Meaning | |---|---|---| SNAKE HTML | pigame/games/snake.html | Override HTML path | MOVE STEPS | 1 | Grid steps per game move | HEADED | unset | 1 → show Chromium | This is the intended product flow: you open Pi; the LLM uses tools . cd /path/to/pigame HEADED=1 pi -e ./.pi/extensions/pigame.ts or: pi auto-loads .pi/extensions/pigame.ts if the project is trusted After code changes: /reload . You should see something like: GameMind ready — LLM tools: game observe, game move, game wait . In Pi: /game connect Playwright opens Chromium and loads pigame/games/snake.html agent-mode , then Start. World is frozen until each game move . Do not open snake.html in your own browser — only the Playwright window is controlled.Optional: /game observe to sanity-check AgentState. Example DeepSeek / any model in Pi : Use only game observe , game move , and optionally game wait . Eat one food: observe → choose a direction → move → repeat. When score increases, stop and report before/after score. Do not reverse 180° from the previous move. Or load the skill: /skill:eat-food . game observe → read player / food / score game move → one step toward food game observe → … … until the model decides the goal is done … | Command | Action | |---|---| /game | Status + help | /game connect | Start Playwright Neon Snake | /game observe | Compact AgentState manual check | /game state | Full AgentState JSON | /game reset | Restart round | /game disconnect | Close browser | /game mock | Switch to mock adapter | /game eat | Test scaffolding — scripted auto-eat | /game survive | Test scaffolding — scripted play until death | | Tool | Role | |---|---| game observe | Eyes | game move | Hands / feet up / down / left / right | game wait | Wait | These commands do not use Pi or an LLM. They run fixed policies eatFoodLoop / surviveUntilEnd to verify Adapter + Perception + I/O. Use them for CI and debugging — not as the agent design. Eat one food score 0 → 10 , headless npm run smoke:eat Same with visible Chromium WSLg npm run smoke:eat:headed Long run until death or stop at a score npm run smoke:survive:headed MAX SCORE=50 npm run smoke:survive:headed | Command | What it checks scaffolding | |---|---| npm run smoke | Mock adapter / AgentState pipeline | npm run smoke:snake | Playwright open + Start + key + screenshot | npm run smoke:perception | CV fixture + live perception | npm run smoke:eat | Scripted steer until score↑ | npm run smoke:survive | Scripted loop until game over or MAX SCORE | npm run smoke: :headed | Same with HEADED=1 | | Variable | Default | Meaning | |---|---|---| HEADED | unset | 1 / true → show Chromium | SNAKE HTML | vendored agent snake | Path override | MOVE STEPS | 1 | Steps per move agent mode | MAX SCORE | none | Survive stops when score ≥ value | MAX STEPS | 50000 | Survive step cap | STEP MS | 200 | Visual delay between smoke moves | HOLD MS | 5000 – 8000 | Keep window open after headed runs | Neon Snake has no clear — only death; MAX SCORE is an artificial stop for tests. . ├── README.md / README.zh-CN.md ├── package.json script proxies ├── docs/ design & schema ├── pigame/ main package │ ├── games/snake.html DEFAULT agent-mode Neon Snake vendored │ ├── games/README.md │ ├── extensions/pigame.ts │ ├── skills/ │ ├── scripts/ smoke: test scaffolding │ └── src/ ├── .pi/extensions/pigame.ts Pi entry └── mini-game-studio-ai-gen/ optional upstream clone SNAKE HTML=… Design docs: docs/README.md /ifoxhz/pigame/blob/main/docs/README.md . | Issue | Fix | |---|---| Missing pigame/games/snake.html | Pull latest pigame — file is in-repo | | Want upstream continuous loop | SNAKE HTML=./mini-game-studio-ai-gen/games/snake.html clone Digiman first | | Snake runs by itself while LLM thinks | You are not on agent-mode HTML; check default path / SNAKE HTML | ENOENT package.json at root | Use root scripts or cd pigame | | No Chromium window | HEADED=1 or :headed ; check WSLg / echo $DISPLAY | | Opened Snake yourself, nothing happens | Only Playwright’s window is controlled | | Pi extension not loaded | pi -e ./.pi/extensions/pigame.ts then /reload | Thanks to Mini Game Studio Digiman/mini-game-studio-ai-gen for the original Neon Snake and HTML game collection.We vendor a patched copy under pigame/games/ for Pi + LLM agent-mode testing paused auto loop + single-step moves .Work in progress MVP: Pi + LLM drives Snake via observe / move . Phase notes: docs/implementation.md /ifoxhz/pigame/blob/main/docs/implementation.md .