// the find
tjmlabs/ColiVara
Colivara is a suite of services that allows you to store, search, and retrieve documents based on their visual embedding. ColiVara has state of the art retrieval performance on both text and visual documents. using vision models instead of chunking and text-processing for documents. No OCR, no text extraction, no broken tables, or missing images.
ColiVara is a document retrieval API built on ColPali — it converts documents to images and uses vision-language model embeddings instead of text extraction, so tables, charts, and layout-heavy PDFs survive the retrieval pipeline intact. It wraps Postgres+pgvector, an external GPU embedding service, and S3 storage behind a REST API with Python and TypeScript SDKs. The target is teams doing RAG over documents where OCR pipelines have failed them.
The ColPali late-interaction approach genuinely outperforms pooled-embedding text pipelines on visually rich documents — the benchmark numbers on InfoVQA (90.1) and the government/healthcare datasets are credible and reproducible via their public eval repo. Using HalfVecs in pgvector is a smart tradeoff: ~2x storage reduction with minimal accuracy loss, and the migration history shows they did this properly rather than bolting it on. The metadata filtering on both collections and documents is more flexible than most vector-search APIs — arbitrary key lookups and has_any_keys without needing a separate structured DB. The separation of the embedding service (ColiVarE) as a modular GPU component means you can swap storage backends or run it serverless without touching the rest of the stack.
The license is Functional Source License (FSL), not Apache or MIT — FSL restricts commercial use of the hosted API and competing products until 2 years after release, which is a real constraint that the README buries in a single line. The self-hosted path requires a GPU with 8GB VRAM minimum for the embedding service, so this isn't a cheap local experiment — you're committing to GPU infrastructure or paying for their hosted API. DocVQA benchmark score (54.8) is notably weak compared to the other datasets and is quietly sandwiched between stronger numbers; if your documents look like DocVQA, the pitch of 'outperforms existing systems' needs scrutiny. The Django/Python API layer is fairly conventional and the test suite, while claiming 100% coverage, tests against a mocked embedding service — so the real-world accuracy of retrieved results in production depends heavily on the GPU endpoint you wire in.