// the find
DaoyuanLi2816/can-i-finetune-this
Estimate whether a Hugging Face model fits and fine-tunes on your local GPU.
A CLI tool that estimates whether a Hugging Face model will fit in your GPU's VRAM for LoRA/QLoRA fine-tuning, before you download 14 GB of weights and OOM on step 1. It models the actual training memory breakdown — not just model load size — and generates ready-to-run training scripts. Aimed at anyone trying to fine-tune LLMs on consumer hardware (12–24 GB cards).
The memory model goes beyond naive weight-size math: it accounts for the logits/cross-entropy chain (which doesn't shrink with gradient checkpointing and can be 4+ GB for 152k-vocab models), fp32 upcasting of embeddings under QLoRA, and optimizer state variants. Coefficients are fitted against real RTX 4080 measurements with the actual numbers published in the repo — estimates land 1–2 GB high rather than promising the moon. The two-layer install (core runs without PyTorch, training deps are optional) means you can use it on a CPU-only machine to plan before touching the GPU box. Tests are pinned against those real measured baselines, so the estimator can't silently drift.
Coverage is narrow: single consumer GPU, causal LM only, Hugging Face stack — no multi-GPU (ZeRO/FSDP), no encoder-decoder, no vision models. The calibration data is all from one RTX 4080; estimates on AMD, older NVIDIA architectures, or cards with different memory bandwidth characteristics are untested extrapolations. The model metadata fetching depends on Hugging Face Hub being reachable, so air-gapped or corporate proxy environments will hit friction fast. No throughput modeling yet, so you can confirm a config fits but can't tell if it'll take 3 hours or 3 days.