cd /news/machine-learning/a-double-dqn-agent-that-plays-snake-… · home › topics › machine-learning › article
[ARTICLE · art-173747] src=dev.to ↗ pub= topic=machine-learning verified=true sentiment=↑ positive

A Double DQN agent that plays Snake, written entirely in KingCobra

A developer built a Double DQN agent that learns to play Snake entirely in KingCobra, their own programming language, using only its standard library with no Python or external dependencies. The agent, a 19→96→96→3 network with 11,523 parameters, reaches a mean score of 33.1 (best 50) over 100 greedy games on a 10×10 board after roughly two minutes of training on one CPU core. Removing the flood-fill free-space inputs drops the mean score from 33 to 21, and the project ships with a live web dashboard served by KingCobra's std.http.

key takeaway

Adding flood-fill estimates of reachable free space after each possible move to the 19-input state raised the Snake agent's mean score from 21 to 33.

by read4 min views1 publishedNov 21, 2026

I built a Double DQN agent that learns to play Snake, and every part of it is written in KingCobra, my programming language: the game, the replay buffer, the neural-network training, the HTTP server and a live web dashboard. It uses only KingCobra's standard library (std.nn, std.http, std.json and std.system). There is no Python and there are no external dependencies.

Repo: https://github.com/DivyanshuSinha136/KingCobra-Snake-DQN

The dashboard, captured from the real app: the trained agent playing, what it sees and wants, and how it learned.

| Score | mean 33.1 (best game50 ) over 100 greedy games on a 10×10 board | | Training | about 2 minutes on one CPU core | | Agent | Double DQN, 3-step returns, Huber loss, gradient clipping, soft target updates, optional prioritized replay | | Network | 19 → 96 → 96 → 3, 11,523 parameters | | Reproducible | one seed controls everything, and two runs with the same seed give bit-identical models |

The model is small on purpose. The interesting part is that a training loop of this kind runs end to end in a language I wrote, on its own interpreter.

You need KingCobra 1.0 (kingcobra on your PATH), and node only for the front-end tests.

git clone https://github.com/DivyanshuSinha136/KingCobra-Snake-DQN
cd KingCobra-Snake-DQN

make serve    # open http://127.0.0.1:8765, a pre-trained model is included
make watch    # or watch it play in the terminal
make train    # retrain from scratch (about 2 minutes on one CPU core)
make test     # 48 KingCobra tests plus front-end tests

No make? Every target is one command: kingcobra run src/main.kc <command> [options], where the commands are train, eval, watch, serve and info.

28 seconds recorded from the real dashboard: the AI plays, a human takes over with the arrow keys, and the AI resumes.

The front-end is a dependency-free canvas page, served by KingCobra's own std.http under a strict Content-Security-Policy. It shows:

You can take over at any time with the arrow keys, and the network keeps showing what it would have done. It also works on a phone.

Human mode: steer with the arrow keys while the network shows what it would do.

What the network wants: one bar per move, with the greedy choice highlighted.

State. The network doesn't see pixels. It gets 19 numbers expressed relative to the snake's heading: danger on the left, straight ahead and on the right, where the food is (ahead, behind, left or right), how many free cells there are before an obstacle on each side, the body length, and a flood-fill estimate of the free space reachable after each possible move. The flood-fill inputs are what teach the agent not to box itself into a pocket. Removing them drops the mean score from 33 to 21.

Learning. For a minibatch of transitions, the target for the action that was taken is:

y = r_n + γⁿ · (1 − done) · Q_target(s', argmax_a Q_online(s', a))

The online network picks the next action and the target network scores it. That's Double DQN, and it removes the over-estimation bias of plain DQN. The whole update is a handful of tensor operations with no per-sample loops, so a batch of 64 costs a fraction of a millisecond. The vectorised target is unit-tested against a one-sample-at-a-time reference to 1e-9.

The greedy evaluation score climbs as epsilon decays. The circled point is the best checkpoint (episode 650), and that's the model that ships.

In the 800-episode run, the evaluation score rises from 3.7 to 36.6. Hyper-parameters and an ablation are in docs/TRAINING.md, and every tunable lives in config/default.json. Unknown keys and nonsensical values are rejected with a clear message instead of silently falling back to a default.

src/
  main.kc        CLI (train · eval · watch · serve · info)
  config.kc      defaults, JSON overlay, --set overrides, validation
  env.kc         Snake rules, rewards, ASCII renderer
  features.kc    state encoder (+ flood fill)
  replay.kc      circular buffer, optional prioritised sampling (sum tree)
  agent.kc       Double DQN: networks, vectorised learning step, save/load
  trainer.kc     training loop, evaluation, checkpointing, metrics
  server.kc      routing, sessions, security headers, static assets
  util.kc        RNG, statistics, helpers
web/             index.html · style.css · app.js
config/          default.json
models/          pre-trained model + model card + curves

The server exposes a small JSON API (/api/state, /api/reset, /api/ai-step, /api/human, /api/model, /api/metrics, /api/benchmark and /api/health). Games are per-session, so several browser tabs never share a board.

make test runs 48 KingCobra tests covering:

There are also unit tests for the front-end's pure functions, and a smoke test that runs the real app.js against a live server with a stub DOM.

env.size but need their own training run. I wanted a project that stresses std.nn the way real work would: a vectorised learning step, a replay buffer, a checkpointed training loop, determinism and a live app on top. If you want the background, my guide to std.nn covers reinforcement learning with DQN and Double DQN on a grid world: https://github.com/DivyanshuSinha136/KingCobra/blob/main/tutorial/KingCobra_std_nn_Complete_Guide_Expanded.pdf

MIT licensed. If you retrain it with other settings, or beat 33.1, tell me the config and I'll link it. A star helps other people find it.

── more in #machine-learning 4 stories · sorted by recency
── more on @kingcobra 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/a-double-dqn-agent-t…] indexed:0 read:4min 2026-11-21 · —