// the find
huggingface/peft
🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
The de-facto standard library for parameter-efficient fine-tuning of large models, supporting LoRA, QLoRA, AdaLoRA, IA3, prompt tuning, and a growing zoo of newer methods. Primarily aimed at ML engineers and researchers who need to fine-tune LLMs or diffusion models without full GPU budgets. Tightly integrated with the Hugging Face ecosystem (Transformers, Diffusers, Accelerate, TRL).
- LoRA support is production-grade: handles adapter merging/unmerging, multi-adapter switching, FSDP/DeepSpeed compatibility, and quantized bases (QLoRA) without much ceremony
- The method breadth is genuinely impressive — VeRA, DoRA, OFT, BOFT, AdaLoRA, IA3, prefix/prompt tuning, and dozens more are all accessible through a unified config interface
- Adapter hot-swapping and multi-adapter weighted inference (e.g., for Diffusers LoRA mixing) work reliably and are well-documented with examples
- Active maintenance with frequent releases; CI covers torch.compile, FSDP, DeepSpeed, and integration tests, so breakage from upstream changes gets caught quickly
- The explosion of methods (40+ at this point) means most non-LoRA methods have shallow test coverage and sparse documentation; using something like VBLoRA or MISS in production is a gamble
- The PeftModel wrapper adds a non-trivial layer of indirection that can break custom training loops, gradient checkpointing interactions, or tools that inspect model internals — debugging these is painful
- No built-in benchmarking or method-selection guidance beyond a comparison Space; you're on your own figuring out whether DoRA or LoRA+ actually helps for your specific task
- Checkpointing behavior has edge cases, particularly around saving/loading with quantization or when multiple adapters are active — the troubleshooting docs exist because people hit these regularly