// the find
SearchSavior/OpenArc
Inference engine for Intel devices. Serve LLMs, VLMs, Whisper, Kokoro-TTS, Embedding and Rerank models over OpenAI endpoints.
OpenArc is a FastAPI server that exposes OpenVINO-accelerated inference (LLMs, VLMs, Whisper, TTS/ASR, embeddings, reranking) over OpenAI-compatible endpoints, specifically for Intel CPU/GPU/NPU hardware. It's for people running local AI on Intel silicon who want an OpenAI-shaped API instead of writing OpenVINO GenAI boilerplate themselves.
Genuinely broad modality coverage through one server — LLM, VLM, Whisper, Kokoro/Qwen3 TTS, Qwen3 ASR, embeddings and reranking all behind the same OpenAI-style routes, which is more than most local inference servers attempt. There's real engineering under the hood: per-request metrics (ttft, prefill/decode throughput, tpot), automatic model unload on inference failure, and three separate backend engines (optimum-intel, openvino-genai, raw openvino) picked per model type rather than one thin wrapper. It also has an actual test suite split into unit and integration tests per backend, plus CI for tests and docs, which a lot of hobbyist inference projects skip entirely.
It depends on nightly openvino-genai wheels to run current models, which means the install is fragile and can break on a schedule you don't control. It's a hard niche: useless unless you're specifically on Intel CPU/GPU/NPU, which caps the audience considerably compared to llama.cpp or vLLM. The README is a badge wall and feature checklist rather than an explanation of how the multi-engine architecture actually routes requests — real docs live on a separate site, so first-contact orientation from the repo itself is thin. It's also effectively a one-person project with a Discord-gated contribution process ('discuss with us on discord... before submitting a PR'), so bus factor and community review depth are both limited.