// the find
karpathy/micrograd
A tiny scalar-valued autograd engine and a neural net library on top of it with PyTorch-like API
micrograd is Karpathy's minimal reverse-mode autodiff engine (~100 lines) plus a tiny MLP library built on top of it, both operating on scalar Values rather than tensors. It exists to teach how backprop actually works, not to train real models.
The whole computation graph and backward pass fit in one readable file, so you can step through every line and understand exactly how gradients flow — no CUDA kernels or tensor broadcasting to obscure the logic. The PyTorch-like operator overloading (+, *, **, relu) makes the API feel familiar despite the tiny implementation. Tests check gradients against real PyTorch, so you can trust the math is correct. The graphviz tracing notebook is a genuinely useful way to see forward values and gradients on the same diagram.
It's scalar-valued, so anything beyond toy MLPs (the moon dataset demo) is computationally out of the question — no vectorization means no real training runs. There's no GPU support and never will be; this is explicitly not for production or even serious experimentation. Test suite requires installing PyTorch just to validate correctness, which is a heavy dependency for a project whose entire point is being dependency-free. The library itself hasn't meaningfully changed in years — most of the recent activity is the microgpt gist linked in the README, not this repo.