// the take
BELLE is a research project for instruction-tuning Chinese LLMs on top of BLOOM and LLaMA, with open training code (DeepSpeed-Chat integration covering full fine-tuning, LoRA, PPO and DPO), large generated instruction datasets, and an eval set scored by GPT-4/ChatGPT. It's aimed at researchers and practitioners building Chinese-language instruct models who want a reproducible training pipeline rather than a single polished model download.
Ships real training code wired into DeepSpeed-Chat for full fine-tuning, LoRA, PPO and DPO, not just a model card. Backs its claims with multiple published technical reports containing actual experiments and numbers (LoRA vs full fine-tune cost/quality tradeoffs, vocabulary extension effects). Releases instruction datasets at real scale (3.5M+ Chinese examples with category labels) on HuggingFace that are usable independently of BELLE's training code. Includes a GPT-4/ChatGPT-based scoring pipeline and ~1k eval set that's a reasonable starting point for comparing other Chinese instruct models.
Base models are LLaMA v1 and BLOOM, both outdated next to current Chinese open models like Qwen or GLM; the LLaMA-derived weights also require XOR reconstruction against your own licensed LLaMA checkpoint rather than a direct download. License is research-only, non-commercial, for both models and datasets, which blocks production use outright. Development has narrowed sharply: the only 2024 activity is Whisper speech models, not the core instruction-tuning work, so the LLM side looks effectively stalled. The authors' own writeups admit the eval set is too narrow to claim parity with ChatGPT, and flag weak performance on math, reasoning and code tasks.
We read the README, the directory tree and a key source file in October 2026. Nobody pays to be listed. How we judge →