// the find
meta-pytorch/torchrec
Pytorch domain library for recommendation systems
TorchRec is Meta's open-sourced library for training recommender systems with massive embedding tables that don't fit on a single GPU. It handles the distributed sharding problem that makes RecSys fundamentally different from other deep learning workloads — your embedding tables might be terabytes, your batch sizes enormous, and your access patterns sparse and unpredictable. It powers DLRM and production models at Meta.
The sharding planner is genuinely useful — it automatically generates optimized partitioning strategies (row-wise, column-wise, table-wise, hybrid) rather than requiring you to hand-tune how tables are split across GPUs. Pipelined training overlaps data transfer, inter-device communication, and computation, which matters a lot when your bottleneck is moving sparse embeddings around. The FBGEMM kernels underneath are well-optimized for the specific access patterns RecSys produces. Dynamic embedding support (via the contrib TDE module) lets you handle unbounded cardinality without pre-allocating fixed-size tables — genuinely hard to build yourself.
The install story is rough: you're pinning to nightly PyTorch builds for CUDA support, which means your environment can break on any given day. The dynamic embedding module lives under contrib/ with a separate build system, so it feels like a second-class citizen despite being one of the most interesting parts. Documentation is thin outside the DLRM example — if your model architecture diverges from that template you're reading source code. It's also built around Meta's infrastructure assumptions; if you're not already thinking at the scale where sharding is necessary, the abstractions add more complexity than they solve.