// the find
xinnan-tech/xiaozhi-esp32-server
本项目为xiaozhi-esp32提供后端服务，帮助您快速搭建ESP32设备控制服务器。Backend service for xiaozhi-esp32, helps you quickly build an ESP32 device control server.
Backend server for the xiaozhi-esp32 AI voice assistant project — handles WebSocket/MQTT connections from ESP32 devices, routes audio through ASR→LLM→TTS pipelines, and manages device sessions. It's a Python + Java + Vue stack aimed at makers who have the companion ESP32 firmware running and want to self-host instead of relying on the author's cloud. 10k stars almost entirely from the Chinese maker community.
The provider abstraction for ASR/LLM/TTS is genuinely well thought out — you can swap in local FunASR, Ollama, or any OpenAI-compatible endpoint without touching the core pipeline. The streaming path (XunfeiStreamASR + qwen-flash + Volcano streaming TTS) shaves ~2.5 seconds off latency compared to the batch path, and the README is honest about that tradeoff. MCP support (client IoT, server-side, and the MCP endpoint protocol) gives real extensibility for home automation use cases. Docker deployment is reasonably solid with separate Dockerfiles for server and web.
The repo explicitly warns it hasn't passed any security audit and should not be deployed to the public internet — that's not a caveat, that's a load-bearing disclaimer for a project that opens WebSocket ports and handles audio from arbitrary devices. The stack is three languages (Python for the core server, Java for the management API, Vue for the UI) with no shared types or contracts between them, which makes contributions and debugging painful. Voiceprint recognition runs locally via 3D-Speaker but the integration docs are thin and there's no clear story for what happens when voiceprint data corrupts or needs migration. The 'minimal install' mode stores state in config files rather than a database, which is fine for one device but falls apart the moment you have more than a handful of agents.