Training a reinforcement-learning agent to play Baloot (and shipping it to Android)

A developer built Arb3a Baloot, a reinforcement-learning agent that plays the Saudi trick-taking card game Baloot, and shipped it as a free, ad-free Android app this week. The bot uses PPO with self-play on a simulator reusing the production server's engine code, with separate bidding and card-play policy heads on a shared trunk, per-round qaid-based reward shaping, and a distillation stage that compresses the best checkpoint into a small on-device network for offline play. Candidates are promoted only after beating the current champion across 960 fixed dealt games with rotated seats, while a slower search-based player that edges the champion network remains a benchmark.

Baloot is a 32-card trick-taking game played in Saudi Arabia by four players in two teams: two contracts Sun and Hokm , two bidding rounds, declarable melds called projects, doubling up to four times, and a scoring system qaid with rounding rules that catch even experienced players out. Building a bot that plays it well turned out to be a much bigger project than the card game itself. This is a write-up of the approach used in Arb3a Baloot Unity client, Node.js server , which shipped on Android this week. The first opponent was a hand-written heuristic: count trump strength, bid Hokm with jack and nine, lead high in Sun. It played legal Baloot but lost to humans, mostly because bidding depends on your partner's likely hand and the dealer position, not only on your own cards. Every hand-coded fix created a new exploitable habit. The encoder produces a 571-dimensional vector per decision: PPO with self-play on a simulator that reuses the server's engine code, so the bot trains on exactly the rules it faces in production. Bidding and card play are two policy heads on a shared trunk. Rewards are shaped per round from the qaid delta rather than only at match end, which roughly halved wall-clock time to a competitive policy. A distillation stage compresses the best checkpoint into a small network that runs on-device for offline play. A candidate is promoted only when it beats the current champion over a fixed set of 960 dealt games with seats rotated, so both sides see the same cards. A search-based player lookahead over sampled hidden hands beats the champion network by a few points but is too slow to run for every seat on the server, so it stays a benchmark. The Android build is a Unity IL2CPP app bundle arm64 + armeabi-v7a . The game is free with no ads. Happy to answer questions about the encoder or the training setup.