// the find
ufal/whisper_streaming
Whisper realtime streaming for long speech-to-text transcription and translation
A streaming wrapper around Whisper that uses a local-agreement policy to emit stable transcription chunks in real time, without waiting for sentence boundaries. Built by researchers at Charles University in Prague, it targets anyone who needs low-latency speech-to-text from a live audio stream — conferencing, live captioning, voice interfaces. The authors themselves now recommend migrating to their newer SimulStreaming project.
Supports four backends (faster-whisper, whisper-timestamped, OpenAI API, mlx-whisper) so you can pick based on your hardware constraints without rewriting integration code. The local-agreement policy is well-founded — backed by a peer-reviewed paper with measured latency numbers (3.3s on unsegmented speech), not just vibes. VAD/VAC integration is first-class, not bolted on, which matters for production use where silence handling kills naive implementations. The module interface is clean enough to embed in a larger system without fighting the code.
The project is explicitly in maintenance mode — the README says it is 'becoming outdated' and points you at SimulStreaming, so you are adopting something the authors have already moved on from. The TCP server transport is bare-bones and single-client; real deployments need to handle concurrent streams, and the recommended WebSocket path punts you to a third-party repo. The entire codebase is essentially one large Python file (whisper_online.py) with no packaging, no tests, and no typed interfaces — fine for experimentation, rough for anything you want to maintain. Installation is a dependency maze: CUDA versions, torch, torchaudio, VAD models, optional sentence segmenters — getting it running on anything but the exact setup the authors used takes non-trivial trial and error.