// the find
dusty-nv/jetson-voice
ASR/NLP/TTS deep learning inference library for NVIDIA Jetson using PyTorch and TensorRT
jetson-voice is an on-device speech and NLP inference library targeting NVIDIA Jetson hardware (Nano, TX2, Xavier). It wraps NeMo-trained models—QuartzNet for ASR, DistilBERT for NLP tasks, FastPitch+HiFiGAN for TTS—and deploys them via TensorRT for GPU-accelerated inference on ARM. The audience is makers and embedded ML engineers building voice assistants or edge AI pipelines without cloud round-trips.
TensorRT optimization is the real selling point—the TTS demo shows 23x real-time factor after the first run, which is genuinely useful on a Nano's limited GPU. The model coverage is broad for a single library: streaming ASR, keyword spotting, VAD, intent/slot, NER, QA, and TTS all under one API. Docker-first distribution with pre-built JetPack-versioned images eliminates the painful dependency hell that usually kills Jetson projects. The test suite actually runs the models end-to-end rather than mocking, so you know what you're getting.
Last pushed February 2024, targeting JetPack 4.x which is EOL—JetPack 6 on Orin-class hardware is current, and TensorRT 10 breaks backward compatibility with the exported engines this library generates. The NLP models are frozen 2021-era fine-tunes with no path to swap in newer models; the DistilBERT intent classifier tops out at a 12-class Google Speech Commands vocabulary, which is too narrow for anything real. There's no Python package on PyPI—you must use the Docker container, which makes embedding this into your own application architecture awkward. The patches directory (manually diffed transformers and NeMo source) is a red flag for long-term maintainability.