finds.dev

srush/GPU-Puzzles

Solve puzzles. Learn CUDA.

View on GitHub

★ 12,554Jupyter NotebookMITupdated Sep 2024

// the take

An interactive Jupyter notebook that teaches GPU programming fundamentals by having you fill in CUDA kernels written in Numba's Python-like CUDA dialect, progressing from element-wise map to shared-memory matrix multiplication. It's aimed at ML practitioners who use GPUs daily but have never written a kernel themselves.

The puzzle progression is well-paced: map/zip, then guards and 2D indexing, then blocks, shared memory, convolution, prefix sum, and finally tiled matmul, each building directly on the last. Numba's cuda.jit lets you write real kernels (threadIdx, blockIdx, shared memory, syncthreads) in plain Python syntax, so you get the actual mental model without fighting a C++ toolchain. The built-in scoreboard (global reads/writes, shared reads/writes per thread) forces you to think about memory traffic as the real cost, not just correctness, which is the part most tutorials skip. Zero setup required - runs in Colab with a GPU runtime.

It's a single notebook with no updates since September 2024 and no visible issue triage; Numba's CUDA target itself is in flux (classic numba.cuda vs. the newer numba-cuda package), so the dependency underneath this could shift under you. You're learning Numba's CUDA dialect, not CUDA C++, so the syntax and debugging experience don't transfer directly to writing .cu files, using cutlass, or working in Triton/CUDA C++ codebases. It stops at basic shared-memory tiling and never touches warp-level primitives, atomics, streams, or multi-GPU work - the stuff that actually bites you once you leave toy kernel sizes. It also pulls in a specific patched fork of a visualization library (danoneata/chalk@srush-patch-1) rather than a maintained release, which is a fragile pin for something meant to keep working years later.

We read the README, the directory tree and a key source file in October 2026. Nobody pays to be listed. How we judge →

search the same ground:

// at a glance

Stars
12,554
Forks
965
Language
Jupyter Notebook
License
MIT
Last commit
1 Sep 2024

// nearby

If GPU-Puzzles isn't quite it.

All repos →