// the find
camenduru/text-generation-webui-colab
A colab gradio web UI for running Large Language Models
A collection of Colab notebooks that let you run quantized open-source LLMs (Vicuna, LLaMA 2, Mistral, CodeLlama, etc.) through oobabooga's text-generation-webui without needing local GPU hardware. Aimed at people who want to experiment with self-hosted LLMs but only have a free Colab T4. Last updated December 2023.
Each notebook is self-contained — one click gets you a running Gradio UI with no local setup. Coverage of GPTQ 4-bit quantization across most relevant 2023 models is thorough; 7B models run comfortably on a free T4, 13B models fit with 4-bit. The repo is honest about WIP status and license ambiguity around commercial Colab Pro use, which is more than most similar projects bother to do. Useful as a reference for the specific pip install incantations and Hugging Face model IDs needed for each model.
This repo is a dead snapshot of mid-2023 — nothing added since December 2023 and marked WIP at the time. All the models listed (Vicuna, early LLaMA 2, CodeLlama 7B) are multiple generations behind; anyone following this today would miss Mistral 7B Instruct v0.3, Llama 3, Gemma 2, and everything after. There is zero actual code — it's just notebooks with shell commands, so there's nothing to fork or extend. Colab's free tier now throttles GPU access aggressively, so the core assumption (free T4 always available) is much shakier than when these were written.