// the find
hiyouga/EasyR1
EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework based on veRL
A fork of ByteDance's veRL that adds vision-language model support for RL post-training (GRPO, DAPO, RLOO, GSPO, CISPO and friends) on Qwen2/2.5/3, Qwen-VL and DeepSeek-R1-distill models. It's for people doing RLHF-style fine-tuning on multimodal or text LLMs who want vLLM-backed rollout speed without hand-rolling the Ray/FSDP plumbing themselves.
HybridEngine plus vLLM's SPMD mode gives it real rollout throughput, not just algorithmic correctness, and it covers a genuinely wide set of RL algorithms (GRPO, DAPO, RLOO, GSPO, CISPO, ReMax) in one trainer rather than forcing you to bolt on each one yourself. The hardware table gives concrete per-model-size GPU memory numbers for full vs. LoRA fine-tuning, and the Docker/Apptainer images pin exact CUDA/vLLM/torch versions, which saves you the usual dependency hell of RL training stacks. Adoption is backed by a long list of published papers (Vision-R1, Seg-Zero, NoisyRollout, GUI-R1, etc.) actually built on top of it, which is a stronger signal than star count. There's a CI test workflow and a license-check test, so it's not just a research dump with no guardrails.
The authors themselves flag that vision-language models don't work with ulysses parallelism yet (listed under 'Known bugs'), which caps long-sequence multimodal training exactly where this project is supposedly differentiated. There's no SFT or inference code here by design — you need LlamaFactory or something else for the steps before and after RL, so this only covers one stage of the pipeline. Dependency pinning is tight and moves fast (transformers>=4.54.0, flash-attn>=2.4.3, vllm>=0.8.3, Docker tags like ngc-th2.8.0-cu12.9-vllm0.11.0 that get superseded), so expect breakage when you try to upgrade any one piece independently. The README leads with an unrelated plug for the maintainer's other project before you get to install instructions, which is just noise to scroll past.