DOOMFORMER: the marine fights a horde of demons under a hellish sky

Neural networks that run on DOOM's game mechanics: voodoo dolls, conveyor belts, lava and doors.

A number is a crowd of voodoo dolls on a conveyor belt. An activation function is a staircase they have to climb. A matrix multiply is dolls running across lava lines, each line raising a room's light level. Attention is dolls squeezing under ceilings that another number has lowered. When the dolls stop, the light levels are read back out of the running game and compared bit for bit with an independent integer reference. One value off by one and the run fails.

Walk a digit, DOOM reads it

The input is a 28×28 floor in the level. Every tile the marine walks on while holding FIRE inks itself, and the level writes that pixel into the brain. He presses READ. 1,355 voodoo dolls run an MNIST network, compare all 45 pairs of digit scores and count the wins. Only the winner's doll gets through, and it lights that digit on the wall. Python advances the clock, then reads back the 784 pixels, the hidden layer, every comparison and the answer, and checks all 248 values against NumPy.

Every digit 0 to 9 in three walked variants: 30 read correctly, all 30 exact. One READ is 5,705 game tics, 76 to 99 s headless. 1,000 MNIST test digits in 14 DOOM games: 966 read correctly, all 1,000 exact. How it works · drawpad.mp4, 16 s, 1.5 MB

doomformer draw               # opens a window: arrow keys to walk, hold Ctrl to draw, Space on the READ switch
doomformer draw --digit 7     # headless: the marine walks a 7

His brain is the level

The marine's brain is compiled into the level he is fighting in. The dots on the right are 1,345 voodoo dolls of him. A level script senses the nearest demon, the dolls run his transformer and pick the action, and the level pulls the trigger. Python advances the clock and reads. 50/50 decisions exact.

The thinking is time-lapsed; the demons are frozen while he thinks. Details · brain_is_the_level_raw.mp4, 53 s, 22.6 MB

Choose your skill level

pip install git+https://github.com/mottopanikeiku/doomformer   # Python 3.11+, Linux or macOS

The first run builds DOOM's script compiler, so you need git, cmake and a C compiler.

SkillCommandWhat DOOM doesTime
I'm too young to diedoomformer helloruns a tiny neural network3 s
Hey, not too roughdoomformer mnistreads handwritten digits~2 min
Hurt me plentydoomformer demoinvents a DOOM level name20 s
Ultra-Violencedoomformer ask "what does iddqd do"answers your question7-11 min
Nightmare!doomformer demo --nightmare106 maps, 56,390 voodoo dolls2-4 min

How it works

  1. your networkPyTorch / NumPy
  2. integersquantised
  3. a DOOM maplanes, gates, lava lines, doors
  4. ViZDoom runs itvoodoo dolls walk
  5. read backsector light levels
  6. EXACTevery value equals the NumPy integer reference

DOOM turns every extra player-1 start into a voodoo doll: a copy of the player that rides conveyors and triggers every line it crosses. An activation of 5 is five dolls. A staircase lets a doll past each step only if the value clears that step's height, which gives ReLU, rounding and clamping in one shape. Sums accumulate in sector light levels, which turn out to be saturating int16 values.

Getting exact results meant measuring the engine. Two floor moves on one sector in the same tic: one is silently lost. A teleport line cancels lines crossed earlier in the same tic. Maps wider than about 34,000 units put things in the wrong room. One value off by one raises MismatchError. The full list is in HOW_IT_WORKS.md and engine_limits.md.

Hall of fame

The bigger experiments, all exact. Numbers and caveats for each are in DEMOS.md and FLY.md.

DOOM dreams of DOOM

A transformer writes DEMON CANYON across 1,594 DOOM tile instances, a Python generator builds the level from the name, and a marine whose policy runs on 55 DOOM games plays it. Halfway along the route 3 of his neurons are closed as doors and he keeps turning right. All 120 decisions are exact against the equally lobotomised reference.

trilogy.mp4, 48 s, 11.5 MB

DOOM plays DOOM

A marine whose policy runs on 55 DOOM games walks DEMON CANYON, the level DOOM named, which he has never seen: doors, two kills, exit. 102 decisions, all exact.

doom_beats_doom.mp4, 37 s, 10.4 MB

DOOM learns

The marine, his arena and his brain are one map in one DOOM game. Weights are ceiling heights, and learning is 156 voodoo dolls of him raising and lowering them. A blank brain taught by an aimbot scores 4 kills in its first life, then 24, 26, 22 and 24. 931/931 decisions exact.

one_game.mp4, 44 s, 19.3 MB

Lobotomy

One 7-pellet shotgun blast to his DOOM brain turns him into a pacifist: 0 kills over 74 decisions, all exact. This lesion was picked from a scan of 3,000 random blasts.

lobotomy.mp4, 29 s, 4.7 MB

The big one

A 4-layer transformer writes UNHOLY STRONGHOLD over 10,166 DOOM tile instances: 208,598 voodoo dolls across 210 compiled tile maps and 1.9 million game tics.

full_brain.mp4, 28 s, 2.6 MB

A fruit fly

A 255-neuron circuit from the FlyWire connectome steers the marine from inside one map: eat, refuse, groom, escape. The level senses, draws the input spikes and moves him; Python advances the clock and reads. 169/169 steps exact.

FLY.md · fly_alive.mp4, 71 s, 5.8 MB

A whole transformer in one map

Embedding, attention, MLP and logits with no Python between layers. Every dot is one of its 3,548 voodoo dolls, marching layer by layer while it writes SHIL. 2,680/2,680 exposed values exact.

Time-lapse. onemap_shil_raw.mp4, 58 s, 6.5 MB

The showreel

All of it in one minute: voodoo dolls on lava, attention gates, the DOOM brain naming a level, the marine playing it, learning, lobotomy, a transformer in one map and his brain inside the level.

showreel.mp4, 56 s, 25.1 MB

Build your own

Compile your own network into a DOOM map and let its outputs open doors, flash lights or shove the marine.

import doomformer as df

net = df.MLP.from_torch(my_torch_mlp, calib_x, levels=8)
print(net.quantization_report(test_x))    # how close the integer model is to your float model
with df.compile(net, actions={"y0": [df.act.door_open(tag=7)]}, world=my_room) as brain:
    y = brain(net.encode(x))              # computed by voodoo dolls, verified exactly
    brain.record([net.encode(x)], "dolls.gif")

The guide is BUILD_YOUR_OWN.md, and there are twelve small runnable examples.