// the find
w3cj/how-llms-work
An interactive app that walks through every stage of the LLM pipeline, from pattern matching to training a transformer from scratch, with working code you can run locally.
An educational app that builds up the entire LLM pipeline as runnable code instead of slides: an ELIZA-style chatbot, a backprop XOR net, a from-scratch BPE tokenizer, Word2Vec skip-gram embeddings, and finally a decoder-only transformer trained with no ML libraries at all. Aimed at developers who want to actually implement attention and backprop by hand rather than read about them, not at people looking for a production LLM toolkit.
The transformer section is the real deal — multi-head causal self-attention, layer norm, Adam, and nucleus sampling all hand-written in TypeScript, with multi-threaded data parallelism via SharedArrayBuffer so it isn't a single-threaded toy. Every section is pinned to the actual paper it implements (Rumelhart 1986, Vaswani 2017, Sennrich 2016, etc.), which is rare discipline for a demo repo and means you can verify the code against the source instead of trusting the author's gloss. The route/hook/component pattern is identical across all five demos, so once you understand one you can navigate the rest without re-learning the architecture.
Training runs in-browser with no GPU path, so 'training a transformer from scratch' is necessarily toy-scale — the README never states vocab size, context length, or training time limits, so you don't know how far this generalizes before you hit it yourself. The directory tree shows no test files anywhere, which is a real gap for hand-rolled backprop and attention math where a sign error or transpose bug would silently produce plausible-looking garbage. The ML logic is woven directly into Hono route handlers and SSE event emission, so pulling out just the BPE or skip-gram implementation to study or reuse means first untangling it from the web server plumbing.