// the take
TensorRT is NVIDIA's inference optimizer and runtime for deploying trained models on NVIDIA GPUs. This repo is the open-source slice of it — plugins, the ONNX parser, and samples — while the actual optimizing engine ships as a closed-source binary you download separately. It's for teams who've already trained a model and need to squeeze out GPU inference latency/throughput in production, not for anyone exploring deep learning generally.
The fused-attention kernel coverage is genuinely deep — hand-tuned variants per SM architecture (75 through 120) and per sequence length/precision combination, which is the kind of low-level work most inference stacks skip. Explicit quantization and strongly-typed networks (replacing the old implicit/weakly-typed APIs) make precision behavior predictable instead of something the optimizer guesses at. It also covers a wide set of import paths — ONNX, Torch-TensorRT, HuggingFace/Optimum, the raw Network Definition API — so it slots into whatever training pipeline you already have instead of forcing one.
You're locked to NVIDIA hardware, full stop — there's no portability story if you ever need to run on anything else. The actual TensorRT engine isn't in this repo at all; you're building against a separately-downloaded proprietary binary matched to a specific CUDA/cuDNN/Python combination, so this OSS layer is scaffolding, not the product. Major versions break compatibility hard — 11.x just dropped IPluginV2, weakly-typed networks, and implicit quantization outright, so anyone upgrading from 10.x is doing a real migration, not a version bump. And productionizing outside a standard x86_64 Linux box (Jetson, DriveOS, QNX) means cross-compilation toolchains and per-target Docker images, which is a real ops tax if you're targeting embedded or automotive.
We read the README, the directory tree and a key source file in November 2026. Nobody pays to be listed. How we judge →