// the find
axolotl-ai-cloud/axolotl
Go ahead and axolotl questions
Axolotl is a YAML-driven fine-tuning framework that wraps HuggingFace Transformers, PEFT, DeepSpeed, and a growing list of custom kernels into a single config-file workflow. It targets ML practitioners who want to go from dataset to trained checkpoint without writing training loops. If you're fine-tuning LLMs regularly, this is the tool most people in the community have converged on.
- The YAML config covers the entire pipeline — dataset preprocessing, training, eval, quantization, inference — so you can reproduce a run exactly by sharing one file, no scattered scripts.
- Multi-GPU and multi-node support is genuinely first-class: FSDP1/2, DeepSpeed ZeRO 1/2/3, tensor parallelism, context parallelism, and expert parallelism are all supported and have working example configs, not just checkbox mentions.
- Model coverage is kept current — Llama 4, Gemma 4, Qwen3, Mistral Medium 3.5, and a dozen others were added in the last few months. The examples directory has tested YAML for each, not just 'it might work'.
- The custom kernel integrations (ScatterMoE, Liger, SageAttention, Flash Attention 4, NVFP4 MoE LoRA) are the kind of thing you'd spend weeks wiring up yourself, and they're exposed through config flags rather than requiring code changes.
- The config schema has grown to hundreds of options and the interactions between them are not always documented — hitting an obscure combination of adapter + quantization + parallelism strategy often means bisecting GitHub issues rather than reading docs.
- Debugging a broken run is painful: the abstraction hides the training loop, so when loss spikes or a gradient explodes you're reading logs from five layers deep and it's not obvious where to intervene.
- The dependency surface is enormous — torch, transformers, PEFT, DeepSpeed, flash-attn, bitsandbytes, and a pile of custom kernels all need to be at compatible versions. The Docker images help, but installing locally on a machine with an existing environment is still a dice roll.
- The multimodal training support is still labeled beta and the loss masking behavior for vision models has had correctness bugs (there's a fix mentioned in the 2026/06 updates), which is the kind of subtle issue that lets training appear to succeed while producing a worse model.