// the find
junyanz/VON
[NeurIPS 2018] Visual Object Networks: Image Generation with Disentangled 3D Representation.
VON is a 2018 NeurIPS research project from MIT/Google that trains a GAN to jointly generate 3D shapes, 2.5D depth maps, and 2D images, letting you manipulate viewpoint, shape, and texture independently. It's for researchers interested in disentangled 3D-aware image generation from the pre-diffusion era. Not a library — a paper artifact.
The disentanglement is the actual contribution: you can swap texture from one object onto another shape at a different viewpoint, which was non-trivial in 2018. The code structure is clean and inherits the well-organized pix2pix/CycleGAN codebase, so it's readable. Custom CUDA kernels for the differentiable renderer are included with source, not just precompiled blobs. Docker fallback is provided for the hairy gcc-version-dependent kernel compilation step.
Pinned to PyTorch 0.4.1, which is ancient — the CUDA kernel build will fail on any modern GPU driver or CUDA toolkit without significant porting work. Tested only on Ubuntu 16.04 with gcc 6.3.0; the RTX series is explicitly unsupported. The dataset is ShapeNet cars and chairs only, so reproducing results on any other object class requires collecting and preprocessing your own 3D data. Six years of inactivity means no one is maintaining this, and diffusion-based 3D generation has lapped it several times over in output quality.