// the find
jcjohnson/cnn-benchmarks
Benchmarks for popular CNN models
A table of forward-plus-backward timings for ten CNN architectures, from AlexNet to ResNet-200, measured in Torch on Pascal Titan X, GTX 1080, GTX 1080 Ti and Maxwell Titan X cards, with CPU baselines for most models. The last push was September 2017, so these numbers describe 2016 and 2017 hardware on cuDNN 5.x, which makes the repo mainly a historical reference or a template for benchmarking your own machine.
Every table row names its GPU and cuDNN version and splits forward from backward time. The raw per-run JSON is committed under outputs/ with an analysis script alongside it, so the tables can be checked rather than taken on trust. The same cards are measured under cuDNN 4, 5.0 and 5.1, which shows how much of a speedup came from library updates alone. The README states the fixed batch size of 16 and 224x224 input up front, and says plainly that ResNet-200 does not fit on the 8GB GTX 1080 instead of leaving a silent gap.
Hardware and software are not held constant. The GTX 1080 and Maxwell Titan X ran on dual Xeon machines, the Pascal Titan X on an i5 with Ubuntu 16.04, and the 1080 Ti on an i7, with CUDA 8.0 RC on some rows and 8.0 final on others. The headline 1.31x to 1.43x ratios therefore mix host differences into the GPU comparison, and the README does not show that they are negligible. There are no iteration counts, warm-up notes or variance figures, so gaps of a few tenths of a percent, such as VGG-16 on the 1080 Ti versus the Pascal Titan X, cannot be read as real differences. The toolchain is dormant: the benchmarks are Lua scripts on Torch 7, linked to fb.resnet.torch and soumith/inception.torch, and the repo is tagged Python even though the driver is Lua. Reproducing the numbers means rebuilding an old Lua and CUDA stack. The line calling the Pascal Titan X 'without a doubt the best card you can get for deep learning' is a 2016 verdict left in place, and nothing covers Volta, which had been announced months before the last push.