finds.dev

NVIDIA/TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

View on GitHubHomepage ↗

★ 13,427C++Apache-2.0updated Oct 2026

// the take

TensorRT is NVIDIA's inference optimizer and runtime for deploying trained models on NVIDIA GPUs. This repo is the open-source slice of it — plugins, the ONNX parser, and samples — while the actual optimizing engine ships as a closed-source binary you download separately. It's for teams who've already trained a model and need to squeeze out GPU inference latency/throughput in production, not for anyone exploring deep learning generally.

The fused-attention kernel coverage is genuinely deep — hand-tuned variants per SM architecture (75 through 120) and per sequence length/precision combination, which is the kind of low-level work most inference stacks skip. Explicit quantization and strongly-typed networks (replacing the old implicit/weakly-typed APIs) make precision behavior predictable instead of something the optimizer guesses at. It also covers a wide set of import paths — ONNX, Torch-TensorRT, HuggingFace/Optimum, the raw Network Definition API — so it slots into whatever training pipeline you already have instead of forcing one.

You're locked to NVIDIA hardware, full stop — there's no portability story if you ever need to run on anything else. The actual TensorRT engine isn't in this repo at all; you're building against a separately-downloaded proprietary binary matched to a specific CUDA/cuDNN/Python combination, so this OSS layer is scaffolding, not the product. Major versions break compatibility hard — 11.x just dropped IPluginV2, weakly-typed networks, and implicit quantization outright, so anyone upgrading from 10.x is doing a real migration, not a version bump. And productionizing outside a standard x86_64 Linux box (Jetson, DriveOS, QNX) means cross-compilation toolchains and per-target Docker images, which is a real ops tax if you're targeting embedded or automotive.

We read the README, the directory tree and a key source file in November 2026. Nobody pays to be listed. How we judge →

// at a glance

Stars
13,427
Forks
2,417
Language
C++
License
Apache-2.0
Last commit
9 Oct 2026

// nearby

If TensorRT isn't quite it.

All repos →