// the find
GeeeekExplorer/nano-vllm
Nano vLLM
A from-scratch reimplementation of vLLM in ~1,200 lines of Python, covering the core continuous batching loop with prefix caching, tensor parallelism, CUDA graphs, and torch.compile. It's aimed at people who want to understand how production LLM serving actually works without reading vLLM's 100k-line sprawl. Not a drop-in vLLM replacement for production use.
The line count is the main selling point — the block manager, scheduler, and model runner are all readable in an afternoon. Benchmark shows it actually matching or beating vLLM throughput on the test hardware, which is surprising for a reference implementation. The vLLM-compatible API surface means you can swap it in for experiments without rewriting call sites. Tensor parallelism is included, not punted to 'future work'.
Only Qwen3 is implemented in the models directory — no Llama, Mistral, Gemma, or anything else, so the model coverage gap is enormous for anyone not on Qwen3. The benchmark is a single model on a single RTX 4070 Laptop with 8GB; scaling claims for larger models or multi-GPU beyond TP=1 are unverified. No OpenAI-compatible HTTP server, so you can't use it as a drop-in serving backend without wrapping it yourself. Last commit was April 2026 and the repo looks close to feature-complete-and-abandoned rather than actively maintained.