{"slug": "i-built-a-neural-network-using-doom-games", "title": "I built a neural network using DOOM games", "summary": "A developer publishing as mottopanikeiku released doomformer v0.1.0, a project that compiles integer neural networks into DOOM levels so the game's voodoo dolls, conveyor belts, lava lines and doors perform the arithmetic. In the MNIST demo, 1,355 voodoo dolls run the network and all 248 computed values — hidden layer, 45 score comparisons, 10 win counts and the one-hot answer — matched a NumPy integer reference at tolerance 0, with all 30 walked digit variants (0-9 in three variants each) read correctly and exactly. A READ pass took 5,705 game tics, or 163 seconds of real-time DOOM at 35 tics/s (79 seconds wall time), and the package installs via pip on Python 3.11+ for Linux or macOS.", "body_md": "**Neural networks that run on DOOM's game mechanics: voodoo dolls, conveyor belts, lava and doors.**\n\n[Watch the films](https://mottopanikeiku.github.io/doomformer/)\n\nA number is a crowd of voodoo dolls on a conveyor belt. An activation function is a staircase they have to climb. A matrix multiply is dolls running across lava lines, each line raising a room's light level. Attention is dolls squeezing under ceilings that another number has lowered. When the dolls stop, the light levels are read back out of the running game and compared bit for bit with an independent integer reference. One value off by one and the run fails.\n\n<sub>The marine's brain is compiled into the level he is fighting in. The dots on the right are 1,345\nvoodoo dolls of him. They run his transformer and choose his next move; the level senses the demons and pulls the\ntrigger. [Watch the film](https://github.com/mottopanikeiku/doomformer/releases/download/v0.1.0/brain_is_the_level_raw.mp4).</sub>\n\nThe input is a 28×28 floor in the level. Every tile the marine walks on while holding FIRE inks itself, and the level writes that pixel into the brain. He presses READ. 1,355 voodoo dolls run an MNIST network, compare all 45 pairs of digit scores and count the wins. Only the winner's doll gets through, and it lights that digit on the wall. Python advances the clock, then reads back the 784 pixels, the hidden layer, every comparison and the answer, and checks all 248 values against NumPy.\n\n```\ndoomformer draw               # opens a window: arrow keys to walk, hold Ctrl to draw, Space on the READ switch\ndoomformer draw --digit 7     # headless: the marine walks a 7 (Python presses his keys in place of a player)\n▄▄▄▄▄▄▄▄▄▄▄▄▄▄\n       ▄████████████████\n       ████▀▀     ▀████▀\n       ▀▀▀        ▄███▀\n                  ████\n                  ███\n                  ███\n                  ███\n                 ████\n                 ███▀\n                 ████\n                 ▀███\n\nDOOM reads: 7  (right)   -- the winner's doll lit it on the 7-segment display\nEXACT: all 248 values DOOM computed (hidden layer, 45 score comparisons and 10 win counts, each with its twin, and the one-hot answer) equal the NumPy integer reference computed from the 784 pixel ports DOOM holds (tolerance 0); one display firing\npixels: the 784 ports equal the level's canvas (the tiles the pen inked)\nREAD pass: 5,705 game tics = 163 s of real-time DOOM at 35 tics/s, 79 s wall | Python sent 58 commands, 29 readout scripts each followed by the ack script it waits for (0 waited more than one advance), and nothing else (0 rejected)\n```\n\nEvery digit 0 to 9 in three walked variants: 30 read correctly, all 30 exact. In a window DOOM runs in real time, so a\nREAD takes 2¾ minutes; headless it takes 76 to 99 s. [How it works](https://github.com/mottopanikeiku/doomformer/blob/main/docs/drawpad.md) · [film](https://github.com/mottopanikeiku/doomformer/releases/download/v0.1.0/drawpad.mp4)\n\n```\npip install git+https://github.com/mottopanikeiku/doomformer   # Python 3.11+, Linux or macOS\n```\n\nThe first run builds DOOM's script compiler, so you need `git`, `cmake` and a C compiler.\n\n| Skill | Command | What DOOM does | Time | \n|---|---|---|---|\n| I'm too young to die | `doomformer hello` | runs a tiny neural network | 3 s | \n| Hey, not too rough | `doomformer mnist` | reads handwritten digits | ~2 min | \n| Hurt me plenty | `doomformer demo` | invents a DOOM level name | 20 s | \n| Ultra-Violence | `doomformer ask \"what does iddqd do\"` | answers your question | 7-11 min | \n| Nightmare! | `doomformer demo --nightmare` | 106 maps, 56,390 voodoo dolls | 2-4 min | \n\nTimes are from a 16-core Linux machine running 8 DOOM games at once.\n\n## **I'm too young to die**: what you'll see\n\n```\ntiny integer MLP 4-6-5-3 compiled into one DOOM map: 60 voodoo dolls, 1222 sectors\ninput 1/5  x=[-71, -82, 64, 37] -> y=[11, 4, -15]  EXACT (3 outputs + 22 hidden readouts equal the integer reference; 488 game tics)\n...\n5/5 EXACT\n```\n\n## **Hey, not too rough**: DOOM reads handwritten digits\n\n784 pixels → 64 → 45 comparisons → 10 win counts → the answer, all in one DOOM map: 1,355 voodoo dolls and 158,550 sectors. Each ink pixel releases a doll that runs across the hidden layer's weight lines. Dolls then compare every pair of digit scores, and only the digit that wins all nine of its comparisons gets a doll through. DOOM picks the digit and Python checks it. In DOOM the network read 99 of the first 100 MNIST test digits correctly and 966 of 1,000 sampled from all 10,000, every one of them exact. On all 10,000 the same integer network gets 97.16 %.\n\n``` bash\n$ doomformer mnist --digits 2 --seed 1\n\n          ███\n          ████                      ▄▄▄█████████▄▄\n           ████                     ▀█████▀▀▀▀▀████\n            ████                             ▄▄███▀\n             ███▄                       ▄▄▄███████▄▄\n             ████                       ████▀▀▀▀▀▀█████\n              ████                                  ▀███\n               ████                      ▄▄         ▄██▀\n               ▀████                     █████▄▄██████▀\n                ▀██                        ▀▀▀▀▀▀▀▀▀▀\n\nDOOM reads: 1  right            DOOM reads: 3  right\ntest #46                        test #51\n\n2/2 read correctly | 2/2 EXACT: every value DOOM computed (248 per digit: hidden layer, comparisons, win counts and\nthe one-hot answer) equals the NumPy integer reference (tolerance 0); Python only compared | 5,696 game tics per digit | 88 s\n```\n\nTo read your own digit from a photo or drawing: `doomformer mnist --image my7.png`.\n\n## **Hurt me plenty**: a transformer names a level\n\nA transformer trained on 136 official DOOM level names, split into 38 small maps that a pool of DOOM games runs. Python copies the integers between games and picks each letter from the logits DOOM computed.\n\n```\nDOOM wrote: 'GHEER'   (NOT in its training corpus: the model made it up)\n  174 tile runs in 8 parallel DOOM games | 10,462 voodoo dolls in the tile maps |\n  6,902 values read back and compared with the NumPy integer reference (tolerance 0) | 15 s\nEXACT: every value DOOM exposed equals the integer reference, and the reference writes the same name\n```\n\n`--seed 3`, `--seed 4` and so on give other names. `--one-map` puts a whole transformer into a single map (embedding,\nattention, MLP and logits with no Python between layers), at about 90 s per letter.\n\n## **Ultra-Violence**: ask DOOM a question\n\nA 2-layer transformer trained on 189 facts about DOOM. Voodoo dolls across a few thousand DOOM game instances compute every value, and every value is checked. The first letter takes a few minutes and the rest arrive about every 20 seconds. Some answers, all exact in DOOM:\n\n| You ask | DOOM answers | \n|---|---|\n| what does iddqd do | **GOD MODE** | \n| what is love | **INVULNERABILITY** | \n| what is a voodoo doll | **BERSERK** | \n| can it run doom | **SIXTY SECONDS** | \n\nIt knows its cheat codes. Ask it anything else and you get confident DOOM nonsense.\n\n## **Nightmare!**: the big brain\n\nThe 2-layer, 24,279-parameter transformer trained on 11,341 real DOOM level names, compiled into 106 maps with 56,390 voodoo dolls and 785,460 sectors. With these settings it writes DEMON CANYON, which is not among its 12,934 training names:\n\n```\n106 tile maps (56,390 voodoo dolls, 785,460 sectors) built in 6 s; running them in 8 DOOM game processes ...\nDOOM wrote: 'DEMON CANYON'\n  1,690 tile runs in 8 parallel DOOM games | 56,390 voodoo dolls in the tile maps |\n  120,666 values read back and compared with the NumPy integer reference (tolerance 0) | 135 s\nEXACT: every value DOOM exposed equals the integer reference, and the reference writes the same name\n```\n\nClose the doors in front of one neuron (`L1.m` unit 33) and the same brain writes DEMON DEATH.\n\n| A matrix multiply. Each doll is one unit of an activation, and each lava line it crosses adds a weight to a room's light level. | Attention. Dolls squeeze under ceiling gates that another live value has lowered; counting who gets through is a multiply. | \n| Ablation is a door. Shoot the switch, the door slams, and one neuron's dolls never arrive. | DEMON CANYON, a level built from the name the DOOM brain wrote, played by a marine whose brain also runs on DOOM. | \n\n``` php\nflowchart LR\n  NET[\"your network<br/>(PyTorch / NumPy)\"] --> INT[\"integers<br/>(quantised)\"]\n  INT --> MAP[\"a DOOM map<br/>lanes, gates, lava lines, doors\"]\n  MAP --> RUN[\"ViZDoom runs it<br/>voodoo dolls walk\"]\n  RUN --> READ[\"read back<br/>sector light levels\"]\n  READ --> CHECK{\"== NumPy integer<br/>reference?\"}\n  CHECK -->|\"every value\"| EXACT[\"EXACT\"]\n  CHECK -->|\"off by one\"| ERR[\"MismatchError\"]\n```\n\nDOOM turns every extra player-1 start into a voodoo doll: a copy of the player that rides conveyors and triggers every line it crosses. An activation of 5 is five dolls. A staircase lets a doll past each step only if the value clears that step's height, which gives ReLU, rounding and clamping in one shape. Sums accumulate in sector light levels, which turn out to be saturating int16 values.\n\nGetting exact results meant measuring the engine. Two floor moves on one sector in the same tic: one is silently lost.\nA teleport line cancels lines crossed earlier in the same tic. Maps wider than about 34,000 units put things in the\nwrong room. The full list is in [docs/HOW_IT_WORKS.md](https://github.com/mottopanikeiku/doomformer/blob/main/docs/HOW_IT_WORKS.md) and\n[docs/engine_limits.md](https://github.com/mottopanikeiku/doomformer/blob/main/docs/engine_limits.md).\n\nThe bigger experiments, all exact. Their code is in [`demos/`](https://github.com/mottopanikeiku/doomformer/blob/main/demos), with training in [`training/`](https://github.com/mottopanikeiku/doomformer/blob/main/training) and\nevery trained model in [`models/`](https://github.com/mottopanikeiku/doomformer/blob/main/models). Numbers and caveats for each are in [docs/DEMOS.md](https://github.com/mottopanikeiku/doomformer/blob/main/docs/DEMOS.md) and\n[docs/FLY.md](https://github.com/mottopanikeiku/doomformer/blob/main/docs/FLY.md).\n\n|  | **A transformer named a DOOM level.** DEMON CANYON, written across 1,594 DOOM tile instances.[film](https://github.com/mottopanikeiku/doomformer/releases/download/v0.1.0/trilogy.mp4) | \n|  | **DOOM plays DOOM.** A marine whose policy runs on 55 DOOM games walks the level DOOM named, which he has never seen: doors, two kills, exit. 102 decisions, all exact.[film](https://github.com/mottopanikeiku/doomformer/releases/download/v0.1.0/doom_beats_doom.mp4) | \n|  | **DOOM learns.** Weights are ceiling heights, and learning is voodoo dolls raising and lowering them. A blank brain taught by an aimbot inside one game scores 4 kills in its first life, then 24, 26, 22 and 24. 931/931 decisions exact.[film](https://github.com/mottopanikeiku/doomformer/releases/download/v0.1.0/one_game.mp4) | \n|  | **A fruit fly.** A 255-neuron circuit from the FlyWire connectome steers the marine from inside one map: eat, refuse, groom, escape. 169/169 steps exact.[film](https://github.com/mottopanikeiku/doomformer/releases/download/v0.1.0/fly_alive.mp4) ·[code](https://github.com/mottopanikeiku/doomformer/blob/main/demos/flyloop.py) | \n|  | **A whole transformer in one map.** Embedding, attention, MLP and logits with no Python between layers: 3,548 dolls write SHIL.[film](https://github.com/mottopanikeiku/doomformer/releases/download/v0.1.0/onemap_shil_raw.mp4) | \n\n- **His brain is the level.** A level script senses the nearest demon, 1,345 voodoo dolls of the marine run his\ntransformer and pick the action, and the level pulls the trigger. Python advances the clock and reads. 50/50\ndecisions exact.[film](https://github.com/mottopanikeiku/doomformer/releases/download/v0.1.0/brain_is_the_level_raw.mp4) ·[code](https://github.com/mottopanikeiku/doomformer/blob/main/demos/brainlevel.py)\n- **Lobotomy.** One 7-pellet shotgun blast to his DOOM brain turns him into a pacifist with 0 kills. This lesion was\npicked from a scan of 3,000 random blasts.[film](https://github.com/mottopanikeiku/doomformer/releases/download/v0.1.0/lobotomy.mp4)\n- **The big one.** A 4-layer transformer writes UNHOLY STRONGHOLD over 10,166 DOOM tile instances and 1.9 million game\ntics.[film](https://github.com/mottopanikeiku/doomformer/releases/download/v0.1.0/full_brain.mp4)\n- [The showreel](https://github.com/mottopanikeiku/doomformer/releases/download/v0.1.0/showreel.mp4) .\n\nCompile your own network into a DOOM map and let its outputs open doors, flash lights or shove the marine.\n\n``` python\nimport doomformer as df\n\nnet = df.MLP.from_torch(my_torch_mlp, calib_x, levels=8)   # or df.MLP(...) by hand, or df.MLP.from_numpy(...)\nprint(net.quantization_report(test_x))                     # how close the integer model is to your float model\nwith df.compile(net, actions={\"y0\": [df.act.door_open(tag=7)]}, world=my_room) as brain:\n    y = brain(net.encode(x))                               # computed by voodoo dolls, verified exactly\n    brain.record([net.encode(x)], \"dolls.gif\")             # film the dolls doing it\n```\n\nSpiking networks (`df.SpikingNet`), transformers (` df.Transformer.load(\"doomnames\")`), conv nets (` from_torch` lowers\nConv1d/2d exactly) and learners that update their weights inside the map (`df.Perceptron`) work the same way.\n`df.with_argmax` lets the dolls pick the winning output. `df.Agent` puts the senses and the muscles in the level as well.\nModels too big for one map spread over a pool of DOOM games with `mode=\"tiles\"`, and `doomformer plan` tells you which\none you need.\n\nThe guide is [docs/BUILD_YOUR_OWN.md](https://github.com/mottopanikeiku/doomformer/blob/main/docs/BUILD_YOUR_OWN.md), and there are twelve small runnable\n[examples](https://github.com/mottopanikeiku/doomformer/blob/main/examples). Made something cursed? [Show and tell](https://github.com/mottopanikeiku/doomformer/issues/new/choose).\n\nOne map is one game, and one game uses one core. `df.run_many` gives each input to the next idle game, checks every\npass, and returns the outputs in order. The games run on your machine or on any Ray cluster you start yourself.\n\n1,000 MNIST test digits in 14 DOOM games on one 16-core machine took 2 hours: 966 read correctly, 1,000 exact, and\n5,696,000 game tics, which is 45 hours of real-time DOOM. Every digit's result is in\n[`models/mnist/doom_1000.json`](https://github.com/mottopanikeiku/doomformer/blob/main/models/mnist/doom_1000.json).\n\n```\nys = df.run_many(net, xs, workers=8)                  # 8 DOOM games on this machine\nys = df.run_many(net, xs, backend=\"ray\")              # every CPU of your Ray cluster\npip install \"doomformer[ray]\"\nray start --head                                      # then `ray start --address=HEAD:6379` on the other machines\ndoomformer mnist --all --ray --record mnist.json      # all 10,000 MNIST test digits, every result in one file\n```\n\nBudget one core and 1.25 GB of RAM per game for the MNIST map. Details in [docs/scaling.md](https://github.com/mottopanikeiku/doomformer/blob/main/docs/scaling.md#scaling-out-many-games-many-machines).\n\n| [`doomformer/`](https://github.com/mottopanikeiku/doomformer/blob/main/doomformer) | the toolkit: map compiler, DOOM runtime, exact checks, CLI | \n| [`demos/`](https://github.com/mottopanikeiku/doomformer/blob/main/demos) | the research code: his brain is the level, the walker and level generator, the fly circuit, the learning arena, the cameras | \n| [`training/`](https://github.com/mottopanikeiku/doomformer/blob/main/training) | quantisation-aware training for every model, and the level-name and DOOM-fact corpora with their sources | \n| [`models/`](https://github.com/mottopanikeiku/doomformer/blob/main/models) | every trained model with its training log and evaluation | \n| [`examples/`](https://github.com/mottopanikeiku/doomformer/blob/main/examples) | twelve one-file examples, most under 15 s | \n| [`tests/`](https://github.com/mottopanikeiku/doomformer/blob/main/tests) | 164 tests, 79 of them on real headless DOOM, including tamper controls that must fail | \n| [`docs/`](https://github.com/mottopanikeiku/doomformer/blob/main/docs) | how it works, every demo with its numbers, the engine facts, the toolkit guide | \n\n## `IDCLIP` Is this really DOOM?\n\nIt is [ViZDoom](https://github.com/Farama-Foundation/ViZDoom) 1.3.2, a ZDoom-based engine, with Freedoom's free\nassets, running headless. The maps use ZDoom's UDMF format and ACS level scripts. ACS moves the input gates, resets the\ndolls and copies results out. In the closed-loop levels it also senses (the nearest demon, the canvas tiles the marine\nwalked on) and carries out the action the dolls chose. Every multiply-add and activation is dolls crossing lines, and in\nDRAWPAD, `mnist` and *his brain is the level* so is the final choice.\n\n## `IDKFA` What does Python do, then?\n\nIt compiles the maps, writes the inputs, waits, reads the results and checks them. In DRAWPAD and *his brain is the\nlevel* the level writes its own inputs. For models split over many maps, Python copies integers between games, and for\ntext it picks each letter from the logits DOOM computed. There is no softmax or LayerNorm in DOOM; attention is\nReLU-style integer attention. Training happens offline in PyTorch, except for the learners that update their own\nweights inside the map.\n\n## `IDDQD` How do you know DOOM computed it and not Python?\n\nEvery value is read back from the running game and compared with a NumPy integer reference that never sees the map, with tolerance 0. The tests also try to break it. They raise one weight line in a compiled map from 5 to 6, and the check fails by exactly +1 on exactly the inputs whose dolls cross that line. They make a readout echo a value Python wrote, build maps without the barriers the engine needs, and let a settle check trust ViZDoom's 4,096-object list. Each of these has to be caught.\n\n## `IDSPISPOPD` Is it fast?\n\nNo. One small MLP pass takes 726 ms in DOOM and 11 µs in NumPy, about 70,000× slower. Reading one handwritten digit takes 5,705 game tics: 76 to 99 s headless, 2¾ minutes of real-time DOOM. The demons are frozen while the marine thinks. Speed was never the goal.\n\n## `IDBEHOLD` Does the marine actually learn to play?\n\nHe imitates a scripted aimbot that sees exact demon positions, binned into 86 possible observations. That is supervised imitation. There is no reward-driven RL and no learning from pixels. What matters here is where the network runs.\n\n## `IDDT` Why?\n\nPeople have run DOOM on calculators, pregnancy tests and lawnmowers. This goes the other way: DOOM is the computer.\n\nCode and docs: [MIT](https://github.com/mottopanikeiku/doomformer/blob/main/LICENSE). The shipped level-name lists (Doom Wiki, CC BY-SA 4.0; id Software's source, GPL-2.0)\nand the 100 sample MNIST digits (CC BY-SA 3.0) keep their licences; see [THIRD_PARTY.md](https://github.com/mottopanikeiku/doomformer/blob/main/THIRD_PARTY.md). Freedoom\ngraphics are BSD-licensed. DOOM is a trademark of id Software / ZeniMax, and this project is not affiliated with them.", "url": "https://wpnews.pro/news/i-built-a-neural-network-using-doom-games", "canonical_source": "https://github.com/mottopanikeiku/doomformer", "published_at": "2026-10-21 20:40:03+00:00", "updated_at": "2026-10-21 21:13:29.938317+00:00", "lang": "en", "topics": ["neural-networks", "machine-learning", "artificial-intelligence", "developer-tools"], "entities": ["doomformer", "DOOM", "NumPy", "mottopanikeiku", "MNIST", "Python"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/i-built-a-neural-network-using-doom-games", "markdown": "https://wpnews.pro/news/i-built-a-neural-network-using-doom-games.md", "text": "https://wpnews.pro/news/i-built-a-neural-network-using-doom-games.txt", "jsonld": "https://wpnews.pro/news/i-built-a-neural-network-using-doom-games.jsonld"}}