{"slug": "a-double-dqn-agent-that-plays-snake-written-entirely-in-kingcobra", "title": "A Double DQN agent that plays Snake, written entirely in KingCobra", "summary": "A developer built a Double DQN agent that learns to play Snake entirely in KingCobra, their own programming language, using only its standard library with no Python or external dependencies. The agent, a 19→96→96→3 network with 11,523 parameters, reaches a mean score of 33.1 (best 50) over 100 greedy games on a 10×10 board after roughly two minutes of training on one CPU core. Removing the flood-fill free-space inputs drops the mean score from 33 to 21, and the project ships with a live web dashboard served by KingCobra's std.http.", "body_md": "I built a **Double DQN agent that learns to play Snake**, and every part of it is written in [KingCobra](https://github.com/DivyanshuSinha136/KingCobra), my programming language: the game, the replay buffer, the neural-network training, the HTTP server and a live web dashboard. It uses only KingCobra's standard library (`std.nn`, `std.http`, `std.json` and `std.system`). There is no Python and there are no external dependencies.\n\nRepo: [https://github.com/DivyanshuSinha136/KingCobra-Snake-DQN](https://github.com/DivyanshuSinha136/KingCobra-Snake-DQN)\n\n*The dashboard, captured from the real app: the trained agent playing, what it sees and wants, and how it learned.*\n\n| **Score** | mean **33.1** (best game**50** ) over 100 greedy games on a 10×10 board | \n| **Training** | about 2 minutes on one CPU core | \n| **Agent** | Double DQN, 3-step returns, Huber loss, gradient clipping, soft target updates, optional prioritized replay | \n| **Network** | 19 → 96 → 96 → 3, 11,523 parameters | \n| **Reproducible** | one seed controls everything, and two runs with the same seed give bit-identical models | \n\nThe model is small on purpose. The interesting part is that a training loop of this kind runs end to end in a language I wrote, on its own interpreter.\n\nYou need KingCobra 1.0 (`kingcobra` on your `PATH`), and `node` only for the front-end tests.\n\n```\ngit clone https://github.com/DivyanshuSinha136/KingCobra-Snake-DQN\ncd KingCobra-Snake-DQN\n\nmake serve    # open http://127.0.0.1:8765, a pre-trained model is included\nmake watch    # or watch it play in the terminal\nmake train    # retrain from scratch (about 2 minutes on one CPU core)\nmake test     # 48 KingCobra tests plus front-end tests\n```\n\nNo `make`? Every target is one command: `kingcobra run src/main.kc <command> [options]`, where the commands are `train`, `eval`, `watch`, `serve` and `info`.\n\n*28 seconds recorded from the real dashboard: the AI plays, a human takes over with the arrow keys, and the AI resumes.*\n\nThe front-end is a dependency-free canvas page, served by KingCobra's own `std.http` under a strict Content-Security-Policy. It shows:\n\nYou can take over at any time with the arrow keys, and the network keeps showing what it would have done. It also works on a phone.\n\n*Human mode: steer with the arrow keys while the network shows what it would do.*\n\n*What the network wants: one bar per move, with the greedy choice highlighted.*\n\n**State.** The network doesn't see pixels. It gets 19 numbers expressed *relative to the snake's heading*: danger on the left, straight ahead and on the right, where the food is (ahead, behind, left or right), how many free cells there are before an obstacle on each side, the body length, and a **flood-fill estimate of the free space reachable after each possible move**. The flood-fill inputs are what teach the agent not to box itself into a pocket. Removing them drops the mean score from 33 to 21.\n\n**Learning.** For a minibatch of transitions, the target for the action that was taken is:\n\n```\ny = r_n + γⁿ · (1 − done) · Q_target(s', argmax_a Q_online(s', a))\n```\n\nThe online network *picks* the next action and the target network *scores* it. That's Double DQN, and it removes the over-estimation bias of plain DQN. The whole update is a handful of tensor operations with no per-sample loops, so a batch of 64 costs a fraction of a millisecond. The vectorised target is unit-tested against a one-sample-at-a-time reference to 1e-9.\n\n*The greedy evaluation score climbs as epsilon decays. The circled point is the best checkpoint (episode 650), and that's the model that ships.*\n\nIn the 800-episode run, the evaluation score rises from 3.7 to 36.6. Hyper-parameters and an ablation are in `docs/TRAINING.md`, and every tunable lives in `config/default.json`. Unknown keys and nonsensical values are rejected with a clear message instead of silently falling back to a default.\n\n```\nsrc/\n  main.kc        CLI (train · eval · watch · serve · info)\n  config.kc      defaults, JSON overlay, --set overrides, validation\n  env.kc         Snake rules, rewards, ASCII renderer\n  features.kc    state encoder (+ flood fill)\n  replay.kc      circular buffer, optional prioritised sampling (sum tree)\n  agent.kc       Double DQN: networks, vectorised learning step, save/load\n  trainer.kc     training loop, evaluation, checkpointing, metrics\n  server.kc      routing, sessions, security headers, static assets\n  util.kc        RNG, statistics, helpers\nweb/             index.html · style.css · app.js\nconfig/          default.json\nmodels/          pre-trained model + model card + curves\n```\n\nThe server exposes a small JSON API (`/api/state`, `/api/reset`, `/api/ai-step`, `/api/human`, `/api/model`, `/api/metrics`, `/api/benchmark` and `/api/health`). Games are per-session, so several browser tabs never share a board.\n\n`make test` runs **48 KingCobra tests** covering:\n\nThere are also unit tests for the front-end's pure functions, and a smoke test that runs the real `app.js` against a live server with a stub DOM.\n\n`env.size` but need their own training run.\nI wanted a project that stresses `std.nn` the way real work would: a vectorised learning step, a replay buffer, a checkpointed training loop, determinism and a live app on top. If you want the background, my guide to `std.nn` covers reinforcement learning with DQN and Double DQN on a grid world: [https://github.com/DivyanshuSinha136/KingCobra/blob/main/tutorial/KingCobra_std_nn_Complete_Guide_Expanded.pdf](https://github.com/DivyanshuSinha136/KingCobra/blob/main/tutorial/KingCobra_std_nn_Complete_Guide_Expanded.pdf)\n\nMIT licensed. If you retrain it with other settings, or beat 33.1, tell me the config and I'll link it. A star helps other people find it.", "url": "https://wpnews.pro/news/a-double-dqn-agent-that-plays-snake-written-entirely-in-kingcobra", "canonical_source": "https://dev.to/divyanshusinha136/a-double-dqn-agent-that-plays-snake-written-entirely-in-kingcobra-493h", "published_at": "2026-11-21 11:28:31+00:00", "updated_at": "2026-11-21 11:41:58.005848+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "artificial-intelligence", "developer-tools"], "entities": ["KingCobra", "Double DQN", "DivyanshuSinha136"], "also_reported_by": [], "takeaway": "Adding flood-fill estimates of reachable free space after each possible move to the 19-input state raised the Snake agent's mean score from 21 to 33.", "recommendation": null, "alternates": {"html": "https://wpnews.pro/news/a-double-dqn-agent-that-plays-snake-written-entirely-in-kingcobra", "markdown": "https://wpnews.pro/news/a-double-dqn-agent-that-plays-snake-written-entirely-in-kingcobra.md", "text": "https://wpnews.pro/news/a-double-dqn-agent-that-plays-snake-written-entirely-in-kingcobra.txt", "jsonld": "https://wpnews.pro/news/a-double-dqn-agent-that-plays-snake-written-entirely-in-kingcobra.jsonld"}}