Downloading the model…
Struck-through pieces of your prompt never appeared in the training data, so the model skips them.
Each shaded chunk is one token. Hover for how likely the model thought it was; red underlines are long shots.
A small GPT I trained from scratch on about 750,000 words of children's stories. It runs entirely in your browser: no server, no API, just a few million numbers doing arithmetic on your device.
Struck-through pieces of your prompt never appeared in the training data, so the model skips them.
Each shaded chunk is one token. Hover for how likely the model thought it was; red underlines are long shots.
It's a 4-layer transformer with 4 attention heads, a 128-dimension embedding and a 128-token context window, trained on a slice of the TinyStories dataset. It's a story-completion model, not a chatbot. It only knows how to keep going in the style of simple children's stories, and its grip on plot is loose.
It uses GPT-2's tokenizer, but only a fraction of those 50,257 tokens ever showed up in training, so the page ships just those rows of the embedding and output layers.
The forward pass is about 150 lines of plain JavaScript in a Web Worker (model.js), with a key/value cache so each new word only costs one step. The training code, built up from a bigram baseline, is on GitHub.