// the find
RyanCodrai/turbovec
A vector index built on TurboQuant, written in Rust with Python bindings
turbovec is a Rust vector index with Python bindings that implements Google's TurboQuant quantization algorithm, targeting the RAG use case where you want in-process ANN search without a separate vector database. It compresses float32 vectors 8–16x by quantizing to 2 or 4 bits with no training phase, and ships hand-written SIMD kernels for both ARM NEON and x86 AVX-512.
The no-train-step design is genuinely useful: you can add vectors incrementally and the codebook is derived from math rather than data, so there's no rebuild cycle when your corpus grows. The filtered search implementation is smarter than the typical post-filter approach — it short-circuits entire 32-vector blocks when nothing in the block is in the allowlist, so selective filters are actually fast rather than paying full SIMD cost and throwing results away. The IdMapIndex with O(1) delete-by-id using swap-remove is a practical feature FAISS doesn't give you cleanly. Framework integrations (LangChain, LlamaIndex, Haystack) are drop-in replacements for the in-memory stores, which lowers the adoption barrier considerably.
The benchmark story has a notable gap: 100K vectors at k=64 is a toy scale — most production RAG deployments are at 1M–10M vectors, and the speed numbers don't cover that range. The x86 results are honestly disclosed as trailing FAISS on 2-bit configs, but the margin is glossed over; for users on Xeon clusters (common in enterprise), FAISS FastScan may still win. The calibration freezes after the first add, which means if your first batch of vectors is unrepresentative (a cold-start or a domain shift), recall degrades and there's no way to recalibrate short of rebuilding the index. There's also no support for on-disk indexes or mmap — the entire index lives in RAM, so the headline '4 GB for 10M docs' only helps if you have 4 GB of RAM to spare per process.