// the find
modelscope/3D-Speaker
A Repository for Single- and Multi-modal Speaker Verification, Speaker Recognition and Speaker Diarization
A speaker verification and diarization toolkit from Alibaba's ModelScope team, bundling CAM++, ERes2Net/V2, ECAPA-TDNN, and self-supervised RDINO/SDPN under one roof with reproducible training recipes on VoxCeleb, CN-Celeb, and their own 3D-Speaker dataset. Aimed at researchers who want to benchmark multiple architectures without hunting down six separate repos. ERes2Net-large hits 0.52% EER on VoxCeleb1-O, which is genuinely competitive.
The architecture breadth is the main draw — supervised, self-supervised, and multimodal diarization all in one place with matching training configs, so comparisons are actually fair. ERes2NetV2 at 17.8M params outperforms the much heavier ECAPA-TDNN across most benchmarks, and the paper numbers are reproducible from the published recipes. The audio+video diarization pipeline is a real differentiator: most open toolkits don't fuse visual cues at all, and on the Chinese meeting datasets they beat pyannote.audio by 5+ DER points. ONNX export support means you can actually ship something to production without being stuck on PyTorch at inference time.
Pretrained models live on ModelScope, a Chinese-hosted platform, which adds friction for anyone outside that ecosystem — you're pip-installing modelscope and pulling weights from a CDN that isn't exactly fast everywhere. The entire recipe structure is Kaldi-style shell scripts: `bash run.sh` with stages, `path.sh`, `local/prepare_data.sh` chains. There's no clean Python API you can import; integrating a model into your own pipeline means reading shell scripts to figure out what's actually happening. The diarization benchmarks tell a mixed story — they win on AISHELL-4 and their own meeting sets, but AMI DER is 21.76% vs DiariZen's 15.4%, which is a meaningful gap for anyone working on English meeting data. Coverage skews heavily toward Mandarin; the English pretrained models are limited to VoxCeleb-trained variants with no obvious path to fine-tuning on your own data without cloning the full recipe structure.