// the take
A minimal C++ (and Python) example showing how to run ONNX Runtime inference with CPU and CUDA execution providers, using SqueezeNet/ResNet image classification as the demo task. It's aimed at developers who want a working CMake/Docker reference for wiring up ONNX Runtime in C++ rather than a library or framework.
Gives a real CMake build setup for ONNX Runtime C++ that actually links and runs, which is the part most people get stuck on since the official docs are Python-first. Includes both CPU and CUDA code paths with measured latency output, so you can see the actual speedup from GPU execution on a concrete model. Docker images pin exact ONNX Runtime and CUDA versions, which sidesteps the usual ABI/version mismatch hell with onnxruntime's prebuilt binaries.
It's a single-model, single-input-shape example (squeezenet/resnet18, fixed 224x224 image classification) — anyone with a different input/output topology, dynamic shapes, or multiple outputs has to rewrite most of inference.cpp themselves. No batching, no session options tuning (thread counts, graph optimization level), and no error handling beyond the happy path, so it's not something to lift into production as-is. Last pushed April 2025 with no activity since, and it's a single-maintainer project (Lei Mao) with no tests, so if a future ONNX Runtime release changes the C++ API you're on your own for fixes.
We read the README, the directory tree and a key source file in October 2026. Nobody pays to be listed. How we judge →