// the find
Lynpoint/CyberVerse
Self hosted, real-time digital human agent platform. Build voice-first AI agents with WebRTC, persona memory, tools, RAG, and optional digital-human video.
CyberVerse is a self-hosted platform for building voice-first AI agents with optional real-time talking-head video driven from a single reference photo. It wires together WebRTC, pluggable LLM/TTS/ASR backends, persona memory, RAG, and GPU-accelerated avatar models (FlashHead, LiveAct) into one deployable stack. Aimed at developers who want a Jarvis-style agent they control, not a SaaS black box.
The plugin architecture is well thought out — brain, voice, hearing, memory, and face are all swappable via YAML config, so you can run cheap pure-voice mode without any GPU and bolt on avatar inference later. The PersonaAgent + SubAgent split is the right call: keeping the foreground agent snappy while long-running tasks run async solves a real latency problem that most hobby projects ignore. The hardware benchmark table with explicit RTP math is unusually honest — they tell you a single RTX 4090 can't do FlashHead Pro at 512×512 realtime, which is exactly the kind of thing projects usually bury. WebRTC with embedded TURN plus optional LiveKit SFU gives real deployment flexibility for NAT-heavy environments.
The dependency surface is punishing: Node 18+, Go 1.25, Conda, Python 3.10, FFmpeg with libvpx, protoc toolchain, and then optionally PyTorch 2.8 with CUDA 12.8 — this is not a Saturday afternoon setup and will break constantly across OS updates. The avatar GPU requirements are brutal for most developers (realtime FlashHead Pro needs dual RTX 5090 or RTX PRO 6000), so the headline feature is gated behind hardware almost nobody has. The project skews heavily toward Chinese cloud providers (Alibaba Qwen, Volcengine Doubao) with OpenAI-compatible endpoints as a secondary path — anyone using Anthropic, Gemini, or self-hosted Ollama will be patching their own plugin. There is no testing infrastructure visible in the tree and no CI configuration, so contributions and refactors are flying blind.