finds.dev

EricLBuehler/candle-lora

Low rank adaptation (LoRA) for Candle.

View on GitHub

★ 174RustMITupdated Apr 2025

// the take

LoRA fine-tuning for HuggingFace's Candle framework, written in Rust. Swaps Linear, Conv1d, Conv2d, and Embedding layers with LoRA counterparts via a proc-macro that keeps model struct changes minimal. Aimed at Rust ML practitioners who want to fine-tune models without leaving the Candle ecosystem.

The proc-macro approach is genuinely ergonomic — adding `#[derive(AutoLoraConvert)]` and `replace_layer_fields` to an existing Candle model struct is much less invasive than manually wiring adapter layers. Weight merging is implemented, so inference overhead is eliminated at deployment time. Pre-converted transformer examples (Llama, Mistral, BERT, Falcon, etc.) lower the barrier to entry considerably. The author also built mistral.rs on top of this, so at least one production-grade consumer exists as evidence it actually works.

Weight format is incompatible with peft's safetensors naming, which means you can't import adapters trained in Python or export to Python tooling — a significant portability problem for anyone in a mixed stack. The candle-lora-transformers models are forks of candle-transformers, so keeping them in sync with upstream Candle as architectures evolve is a maintenance burden that's already showing: last commit April 2025, but Candle itself moves fast. No training loop or optimizer is included — you still have to wire that yourself, so 'LoRA fine-tuning in Rust' is more aspirational than turnkey. 174 stars and 33 forks after two years suggests limited adoption outside the author's own mistral.rs project.

We read the README, the directory tree and a key source file in June 2026. Nobody pays to be listed. How we judge →

// at a glance

Stars
174
Forks
33
Language
Rust
License
MIT
Last commit
18 Apr 2025

// nearby

If candle-lora isn't quite it.

All repos →