finds.dev

whitecircle/halo

Halo is an open-source framework built by White Circle for training large language and multimodal models

View on GitHubHomepage ↗

★ 839PythonNOASSERTIONupdated Oct 2026

// the take

Halo is White Circle's open-source training framework for large language and multimodal models, covering everything from pre-training through SFT, preference optimization, and fully async multi-turn RL. It's aimed at ML engineers running real distributed training jobs (single GPU up to multi-node clusters) who want to stay on standard Hugging Face checkpoints rather than adopt a bespoke model format.

Trains native Hugging Face models directly — checkpoints stay standard SafeTensors and load with from_pretrained, so you're not locked into a parallel model implementation the way Megatron-based frameworks force you to be. Parallelism (EP/CP/TP/ETP) is layered onto the existing model via FSDP2/DTensor/DeviceMesh rather than reimplemented per training method, so pretraining, SFT, DPO, and RL all share the same distributed and checkpointing code. Benchmarks are run against a stock TRL baseline on identical hardware/precision/kernels rather than cherry-picked numbers, and the perf stack (FA4, DeepEP V2, Grouped GEMM, bf16 AdamW with stochastic rounding) is current rather than a thin wrapper around someone else's kernels.

Still very new — first public release (1.0.0) was August 2026, so there's limited track record for catching the long-tail bugs that show up months into large training runs. The headline benchmarks assume 8x B300/H100-class hardware; if you're not running on recent datacenter GPUs, most of the throughput story (and some kernel paths like FA4) doesn't apply to you. Parallelism composition is restrictive by design — EP only combines with one of CP/TP/ETP, other pairings are rejected outright, and ETP is explicitly marked experimental, so multi-axis scaling choices are narrower than frameworks with more mature tensor-parallel support. Governance and licensing sit entirely with one company: it's Apache 2.0 plus supplemental commercial terms (revenue threshold, mandatory attribution above it), not a plain OSS license, which is a real consideration if you're building a product on top of it.

We read the README, the directory tree and a key source file in November 2026. Nobody pays to be listed. How we judge →

// at a glance

Stars
839
Forks
42
Language
Python
License
NOASSERTION
Last commit
30 Oct 2026

// nearby

If halo isn't quite it.

All repos →