// the find
fluxions-ai/vui
Real-time voice assistant — WebRTC streaming, faster-whisper ASR, local LLM, Vui Nano (300M) TTS. OpenAI Realtime API compatible. Voice cloning, barge-in, ~9× realtime on a 4090. Apache 2.0.
Vui is a local, real-time voice assistant stack: faster-whisper transcribes your speech, Ollama runs the LLM, and a custom 300M TTS model (Vui Nano) streams audio back — all on your own hardware. It is aimed at developers who want a fully local voice interface without routing audio through a cloud API, and who are comfortable running a 12GB VRAM GPU setup on Linux.
The pipeline architecture is well thought out — three separate OS processes with multiprocessing queues means the TTS worker (GPU-bound) and ASR worker don't block the aiohttp server, and barge-in actually works cleanly rather than being a hack. The OpenAI Realtime API compatibility layer is a genuine convenience: any client already written against OpenAI's websocket spec works without modification. The SQ/WPS conditioning on the TTS model is genuinely useful — biasing toward lo-fi or clean audio by adjusting six independent quality channels is more expressive than a single quality knob, and the training detail about masked conditioning during training means partial use is safe. The thoughts-stream architecture (a second parallel LLM call that routes intent to tools at temperature=0, never blocking the main reply) is a clean solution to the wake-word grammar problem.
The hardware requirement is steep — 12GB VRAM minimum for the full stack, and the README is honest that Apple Silicon support is a work in progress with known rough edges. The voice cloning quality is explicitly acknowledged to be lower than the four fine-tuned presets, which means the headline feature has a ceiling most users will hit immediately. Memories are stored as a flat JSON file at ~/.vui/memories.json with no deduplication or versioning beyond fuzzy matching — fine for personal use but will quietly accumulate contradictory facts at scale. The telemetry is opt-out rather than opt-in, which will surprise self-hosters who assumed running locally meant no outbound data; the README documents it clearly, but the default should be off for a privacy-first local tool.