▶ Play Snake

This Snake has no game code. A neural network draws every pixel you see: the snake, the food, the score bar, the crash, and the rainbow that fills the screen when you win. It also keeps track of the game itself, from one frame to the next. The page around it knows nothing about Snake. It sends your key presses to the network and shows whatever picture comes back.

The network has about 620,000 parameters and fits in a 2.4 MB file. It runs on your own device, on the graphics chip if your browser supports that and on the processor if it does not.

How it works

Eight times a second, the network gets three things: the memory it wrote on the step before, the keys you are holding, and a bit of random noise. It gives back the next picture and new memory for the step after.

Diagram of the network. In, for every cell: memory from the last step (79 numbers), keys (10), position (10), random noise (15) and power on (1), 115 numbers in all. Then four layers that each look one cell further around, from 3 by 3 to 9 by 9 cells. Then two layers that decide per cell, which also get the input again. Out: the next memory (79 numbers), which goes back in for the next step, and the picture, one of 16 colours per cell.
The whole network. The same small network runs on all 32 × 32 cells at once.

Everything about the game is in that memory: where the snake is, which way it is heading, the score. The page hands it back without reading it, so the network carries the whole game on its own. The noise is what makes food show up in random places, at about the rate the real game would place it.

How I taught it

First I wrote Snake the normal way, as a small program. I call it the teacher. All it does is take a board and the keys being pressed, and work out the next board.

Then I had a simple bot play lots of games against the teacher: steering, eating, crashing, restarting. Every step of every game became one training example. This frame and these keys go in, the next frame comes out. I also made up boards the bot rarely reaches, like long snakes curled into odd shapes.

Diagram: the teacher, Snake written as a normal program, makes training examples. One example is a frame plus the up key, and the next frame, where the snake has turned up towards the food. The network learns to predict the next frame.
The teacher plays Snake and records every step. The network learns to predict what comes next.

The network trained on those pairs until it could predict the next frame by itself. In the last rounds it also played games of its own, and the teacher corrected every step it got wrong. That is how it learned to recover from its own small mistakes.

When you play, your browser only gets the network. The teacher stays on my laptop.

What made it hard

The network only looks at a small patch around each cell, about four cells in every direction. So every rule in the teacher had to work with just that view. That rules out the usual way to write Snake, with a list of where each piece of the snake is. To remove the tail, you would need to know where the head is, and that can be far away.

So in the teacher, every snake cell holds a number. The head has the highest, the tail has 1, and a snake that is 7 long counts 7 down to 1. Each step every number drops by one, the 1 becomes empty, and the cell in front of the head gets the top number. The tail disappears without anything having to move it, and no cell needs to see further than its neighbours. Food works the same way: if the cell in front of the head holds food, it gets one more, and the snake ends up one cell longer.

A snake four cells long, numbered 1 at the tail to 4 at the head. Moving one step right, the old tail cell empties and the numbers 1 to 4 move one cell along. Eating, the head moves onto the food and gets 5 instead of 4, one more than the old head.
A snake of four, moving one step and eating.

Those numbers were part of every training example. So the network learned to keep them in its own memory and to count them down itself.

Keeping score was the hardest part by far. My first versions played well until the first meal. After that the snake grew without eating, lost its head, or the score bar kept counting after a crash. When I measured where things went wrong, more than nine in ten games broke on the score first.

The network’s memory holds more than you see on screen. In the first design, the score was a number kept in every cell near the snake’s head. Each step, a cell copied the highest score among its neighbours. For the network, copying a 2 and copying a 7 are different patterns, and it has to learn each one on its own. Low scores were in lots of training examples, so it learned those. Higher scores were rare, so it never did. From the fourth meal on it kept the old score, and the rest of the game fell apart around it.

I first tried training for longer, but that didn’t fix it, so I changed how the score works. In the new design the memory doesn’t hold the score as a number anywhere. A meal leaves one invisible dot in the memory. Each step the dot drops two rows, and when it reaches the score bar it slides along until it lights the next empty spot. The lit spots are the score. A dot moves the same way at a score of 2 or 20, so the network only has to learn it once. The price is a small delay: the bar shows a meal up to about two seconds after you eat it.

Each training example is still only one step. The network never learns the whole fall, only “if a dot is two rows above me, I hold it now”. It gets its memory back every step, so repeating that one rule carries the dot all the way down.

Where this could go

Program less, show more. For Snake I wrote the whole game as the teacher. I don’t think that will always be needed. You could program the core rules once, train a network on them, and then change the game by example. Want a new level, a faster snake, a second kind of food? Show the network some frames, the keys pressed and the frame that should come next, and fine-tune. I already work a bit like this: when I change the game, the network does not start from zero. It keeps what it knows and learns the difference.

It is a world model. What this network does has a name in AI research: a world model. Given the state of the world and an action, it predicts what happens next. Google’s GameNGen simulates DOOM this way, and DeepMind’s Genie 3 generates whole worlds you can move through. Those learn from huge amounts of footage and need serious hardware, while this one learned from a small program and runs on a phone. That makes it a cheap place to try out ideas that would cost a fortune at that scale.

Many games, one network. This is the idea I find most exciting. Train one network on lots of small games like this, all with the same inputs and the same colours. Each game then becomes a place in what the network has learned. Turn the knobs between those places and you could play what lies in between: something with the movement of one game and the rules of another, or a neural hybrid that is none of the training games and that nobody ever programmed.

Diagram: three small games, Snake, Pong and a brick breaker, each a place in what one network learned. Dashed lines lead from them to a game in the middle where a snake moves on a Pong court, labelled a hybrid nobody programmed.
Each game is a place in what the network learned. In between are games nobody wrote.

Play Snake and see how far you get.