{"slug": "show-hn-how-to-beat-pac-man-with-a-text-classifier", "title": "Show HN: How to beat Pac-Man with a text classifier", "summary": "A developer built a Pac-Man agent that reaches an average of 9.8 ghost kills per game using Jeffy, an open-source CPU text classifier, instead of an LLM, according to the Jevman benchmark project on GitHub. The agent parses the game's structured junction descriptions — roughly 37 unique patterns containing pellet distances, ghost positions, traps and danger flags — into numeric features with regex, then trains a logistic regression head on thousands of junctions labeled by a hand-tuned scoring function. Training takes under a second and the trained model returns a direction in 3ms via Jeffy's predict_features() endpoint.", "body_md": "[Jevman](https://github.com/opper-ai/jevman-benchmark) is a Pac-Man benchmark where AI agents play through structured text descriptions. At each junction, the game tells you what's in each direction (pellet distances, ghost positions, traps) and you pick which way to go.\n\n  The top entries use LLMs. I used [Jeffy](https://github.com/nicobrenner/jeffy), an open-source text classifier that runs on CPU.\n\nAt every junction, the game sends one text description per available direction:\n\n```\nGo left: nearest pellet 3 steps away; 5 pellets in the next corridor;\nnearest dangerous ghost 12 steps away, moving away; you reach the\nnext junction in 4 steps, 6 steps before any ghost\n\nGo right: nearest pellet 1 step away; 2 pellets in the next corridor;\nDANGER: Blinky in this corridor\n\nGo up: nearest pellet 8 steps away; 0 pellets in the next corridor;\nTRAP: Pinky can reach the next junction in 3 steps, you need 5\n```\n\nThere are about 37 unique patterns in these descriptions. All of them contain numbers. All of them are parseable with regex.\n\nThe game descriptions are structured text, so I parse them into numeric features with regex.\n\n``` python\ndef parse_features(description):\n    f = {\"pellet_dist\": 99, \"ghost_dist\": 99,\n         \"ghost_approaching\": 0, \"has_trap\": 0,\n         \"has_danger\": 0, ...}\n\n    for part in description.split(\";\"):\n        m = re.search(r\"nearest pellet (\\d+) steps? away\", part)\n        if m: f[\"pellet_dist\"] = int(m.group(1))\n\n        m = re.search(r\"nearest dangerous ghost (\\d+) steps? away\", part)\n        if m: f[\"ghost_dist\"] = int(m.group(1))\n\n        if \"TRAP:\" in part: f[\"has_trap\"] = 1\n        if \"DANGER:\" in part: f[\"has_danger\"] = 1\n    return f\n```\n\nTo train a classifier you need labeled examples. I wrote a scoring function with hand-tuned weights and played a few hundred games with it. Each junction becomes one training example: the parsed features are the input, the direction the scorer picked is the label.\n\n``` python\ndef score_direction(description):\n    f = parse_features(description)\n    score = 0.0\n\n    # Survival: avoid ghosts, traps, danger corridors\n    if f[\"has_danger\"]: score -= 100\n    if f[\"has_trap\"]:   score -= 80\n    if f[\"ghost_dist\"] < 99:\n        threat = max(0, 20 - f[\"ghost_dist\"]) * 3\n        if f[\"ghost_approaching\"]: threat *= 2.5\n        score -= threat\n\n    # Pellet collection\n    score += max(0, 12 - f[\"pellet_dist\"])\n\n    # Hunt frightened ghosts (200-1600 points each)\n    if f[\"has_frightened_ghost\"]:\n        score += 40 + max(0, 15 - f[\"frightened_ghost_dist\"]) * 3\n\n    return score\n```\n\nA few hundred games produce thousands of labeled junctions, enough to train on.\n\n[Jeffy](https://github.com/nicobrenner/jeffy) is a text classifier that runs on CPU. It supports feature-based classifiers, which is exactly what we need here: the features are already numbers, no embeddings required.\n\n```\n# Train a classifier on the collected game data\njeffy train --task pacman --features pellet_dist,ghost_dist,ghost_approaching,has_trap,has_danger,... \\\n            --data training_games.jsonl\n\n# Test accuracy on held-out games\njeffy eval --task pacman --data test_games.jsonl\n```\n\nTraining takes under a second. The trained model is a logistic regression head over the numeric features. At game time, Jeffy's `predict_features()` endpoint takes the parsed feature vector and returns a direction in 3ms.\n\n  When a power pellet activates, frightened ghosts are worth 200, 400, 800, and 1600 points in sequence. The scoring function shifts from pellet-chasing to ghost-hunting: `+40` base plus a distance bonus for nearby frightened ghosts. Average 9.8 ghost kills per game.\n\n```\n# Install Jeffy\npip install jeffy\n\n# Start the Jeffy endpoint (serves the trained pacman classifier)\npython3 jevman-endpoint.py 8787\n\n# Clone the benchmark and run 24 games\ngit clone https://github.com/opper-ai/jevman-benchmark\ncd jevman-benchmark && npm install\nnpm run submissions -- --model jeffy --endpoint http://localhost:8787\n```\n\nEach game takes 2-5 seconds. Total run: ~90 seconds for all 24 games. The endpoint parses each junction's text into features, feeds them to the Jeffy classifier, and returns a direction.\n\nAn LLM reading \"nearest dangerous ghost 5 steps away, coming toward you\" does the same thing the regex does: extracts the number 5 and the flag \"coming toward.\" It just does it with 7 billion parameters instead of one regular expression, and takes 650ms instead of 3ms.\n\nThe Jeffy classifier learns the same decision boundaries as the hand-tuned scoring function, but from examples instead of manually chosen weights. Once trained, it generalizes to junction configurations it hasn't seen before. And because it's logistic regression over a handful of numeric features, it runs on any CPU in 3ms.\n\n  This is the classifier sweet spot: when the answer is a pattern in the text, not reasoning about the text. Pac-Man with structured descriptions is a classification problem. [The PR is open](https://github.com/opper-ai/jevman-benchmark/pull/18), the CI will verify the results.", "url": "https://wpnews.pro/news/show-hn-how-to-beat-pac-man-with-a-text-classifier", "canonical_source": "https://jeffyclassify.com/blog/pacman/", "published_at": "2026-10-12 15:25:13+00:00", "updated_at": "2026-10-12 15:49:57.473218+00:00", "lang": "en", "topics": ["machine-learning", "artificial-intelligence", "ai-agents", "developer-tools"], "entities": ["Jeffy", "Jevman", "Pac-Man", "GitHub", "nicobrenner", "opper-ai"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-how-to-beat-pac-man-with-a-text-classifier", "markdown": "https://wpnews.pro/news/show-hn-how-to-beat-pac-man-with-a-text-classifier.md", "text": "https://wpnews.pro/news/show-hn-how-to-beat-pac-man-with-a-text-classifier.txt", "jsonld": "https://wpnews.pro/news/show-hn-how-to-beat-pac-man-with-a-text-classifier.jsonld"}}