// the find
mlfoundations/open_clip
An open source implementation of CLIP.
An open source reimplementation of CLIP that's grown into a general multimodal training framework — CoCa, MaMMUT, SigLIP, CLAP audio-text, and variable-resolution 'NaFlex' models all live in the same codebase now. It's for people training or fine-tuning contrastive/generative vision-language models at scale, not for someone who just wants a stable one-line inference API.
Pretrained model coverage is genuinely huge — hundreds of configs spanning ResNet, ViT, ConvNeXt, SigLIP, CLAP, MobileCLIP — and the training stack has actually been run on 1024 A100s with SLURM, so the multi-node code paths aren't theoretical. The decision to kill the torch.jit loading path for original OpenAI weights (routing through HF Hub instead) closes a real arbitrary-code-execution hole rather than just adding a feature. The new model.traits system replacing name-sniffing for NaFlex/variable-text behavior is a legitimate fix to what was clearly a pile of fragile string-matching hacks before.
The main branch changelog in the README is close to a thousand words of breaking changes, renamed flags, and shifted positional args — by the maintainers' own admission the refactor's scope outgrew itself, and that's a bad sign for API stability. Anyone with code built against train_one_epoch/evaluate, tuple-based dataloader batches, or encode_text positional args has a real rewrite ahead of them, not a changelog skim. The default --precision silently flipped from amp to amp_bf16, which changes training numerics for anyone upgrading in place without reading the fine print. Stability now requires pinning to a separate v3 branch indefinitely since main is explicitly not release-stable, which fragments which pretrained weights and scripts actually work together.