// the take
Firefly is a Chinese-first LLM fine-tuning toolkit supporting full-parameter training, LoRA, and QLoRA across a wide range of open-source models. It targets researchers and practitioners who want to fine-tune models like Qwen2.5, Llama3, or Mixtral without writing training boilerplate from scratch. The project is heavily oriented toward Chinese NLP use cases but works for English too.
Config-file-driven training is a genuine usability win — you swap a JSON file to change model, mode, and task type without touching code. The per-model chat template alignment in component/template.py is the kind of tedious-but-necessary work that saves real debugging time. QLoRA results on Open LLM Leaderboard are actually published and competitive — firefly-mixtral-8x7b at 70.16 beats Yi-34B-Chat, which is a credible benchmark, not a cherry-picked internal number. Unsloth integration for memory reduction (42% on Llama3-8B) is practical for anyone training on consumer hardware.
Last commit October 2024 — no Llama 3.1/3.2, no Qwen2.5 72B, no Phi-4, no DeepSeek-V3 support. The dependency story is a mess: different model families require manually uninstalling packages (xformers for Baichuan2, flash-attn for Qwen), which will burn time on any fresh setup. No experiment tracking integration out of the box — TensorBoard only, no W&B or MLflow hooks. The 1024-token training length hard-coded across most configs will limit anyone doing long-context fine-tuning without manually editing every config file.
We read the README, the directory tree and a key source file in June 2026. Nobody pays to be listed. How we judge →