// the find
lllyasviel/ControlNet
Let us control diffusion models!
This is the original implementation of ControlNet, the technique that lets you condition Stable Diffusion 1.5 on edges, depth, pose, segmentation maps, and similar structured inputs instead of just text. It's aimed at people who want to understand or reproduce the method itself, not people who want a ready-to-integrate library — the README says outright that ControlNet 1.1 (a separate repo) supersedes this one.
The zero-convolution / locked-copy-trainable-copy architecture is genuinely well explained, with the authors walking through why zero-initialized weights don't kill gradient flow — rare for a research repo to bother with that. It ships nine working Gradio demos (canny, MLSD, HED, scribble, pose, segmentation, depth, normal map), so you can validate the idea end-to-end without writing any integration code. It's the reference implementation behind the whole ControlNet ecosystem (A1111 plugin, diffusers port), so behavior here is the ground truth when other implementations disagree.
The repo is effectively frozen — last push Feb 2024, and the README itself tells you to go use the ControlNet-v1-1-nightly repo instead, so anything you build against this is building against a dead end. There's no package boundary at all: annotator/ vendors full copies of other projects (mmcv, mmseg, uniformer, midas, openpose) directly into the tree rather than pinning them as dependencies, which makes upgrades and license auditing painful. Model weights have to be manually downloaded from Hugging Face and hand-placed into specific folders with no checksum or fetch script. The interactive scribble UI is described by the authors' own README as 'buggy' and 'difficult to customize' — it's a proof-of-concept, not something to build a product on.