// the take
FastChat is lm-sys's open platform for training, serving, and evaluating LLM chatbots, and it's the actual engine running Chatbot Arena (lmarena.ai) with real traffic across 70+ models. It's for people who want to self-host multiple open or API-backed LLMs behind one OpenAI-compatible endpoint, or reproduce the Vicuna fine-tuning recipe.
The OpenAI-compatible REST API means you can point existing openai-python code at it and swap backends without rewriting client code. The controller/model_worker/gradio_web_server split lets you register several model workers across GPUs and load-balance requests independently, which is the right shape for running more than one model at once. It isn't locked to a single inference engine — there's adapter support for vLLM, exllama v2, GPTQ, AWQ, and SGLang depending on what hardware and throughput you need. And unlike most research-release repos, this one has actually taken 10M+ real chat requests in production via Chatbot Arena, so the serving path is battle-tested, not just a demo.
The project's center of gravity has moved to being the plumbing behind lmarena.ai's research output (Elo leaderboard, battle datasets) rather than an actively developed general-purpose serving framework — the news log stops at March 2024. The default model worker built on huggingface/transformers is explicitly called out in the docs as slow; to get real throughput you're pushed to bolt on vLLM or SGLang anyway, so the out-of-box serving story isn't production-grade on its own. The controller-based routing architecture predates the gateway patterns vLLM/SGLang now build in natively, so you're taking on more operational surface (running and monitoring a separate controller process) than tools designed around that from the start. And the fine-tuning path is tightly coupled to the exact Vicuna recipe — the ShareGPT training data isn't released, hyperparameters are specific to Llama-2-era models, and there's no real support for current PEFT/LoRA-first workflows people expect now.
We read the README, the directory tree and a key source file in October 2026. Nobody pays to be listed. How we judge →