finds.dev

ekimetrics/adaptive-chunking

Adaptive Chunking: automatically select the best chunking method per document for RAG. Accepted at LREC 2026.

View on GitHub

★ 399PythonMITupdated Jul 2026

// the take

Adaptive Chunking (Ekimetrics, LREC 2026) picks the best chunking strategy per document in a RAG pipeline by scoring several chunking methods against five intrinsic quality metrics — no labeled data needed. It's aimed at teams building RAG over mixed document sets (legal, technical, reporting) who are tired of one-size-fits-all recursive splitting.

Both axes are genuinely pluggable — you can register your own chunking method or your own metric function, not just swap config values. The intrinsic metrics (size compliance, cohesion, coreference preservation, block integrity) let you evaluate chunking quality without running a full RAG eval loop, which is the actual hard problem in chunking. Ships three PDF parsing backends (Docling, PyMuPDF, Azure DI) behind one interface, so you're not locked into one. The paper reproduction pipeline is resumable per-document, which matters when a 9-hour local metrics run gets interrupted.

This is paper-release code, not a packaged library — no PyPI listing, install is git clone + pip install -e, and the API surface will likely shift. The coreference metric depends on maverick-coref under CC BY-NC-SA 4.0 and the lightweight PDF parser extra pulls in AGPL/commercial-licensed pymupdf4llm, so anything beyond the default Docling path drags in licensing you need to vet before shipping commercially. Full metric computation is ~9 hours locally on an RTX 4090 unless you pay for the Jina API, and the semantic baseline needs a GPU with flash-attention — this isn't a drop-in CPU utility. RAG evaluation requires hundreds of OpenAI calls, so reproducing or extending the full benchmark has real ongoing API cost, and there's no indication yet of how the project handles malformed or adversarial PDFs in production.

We read the README, the directory tree and a key source file in November 2026. Nobody pays to be listed. How we judge →

// at a glance

Stars
399
Forks
44
Language
Python
License
MIT
Last commit
6 Jul 2026

// nearby

If adaptive-chunking isn't quite it.

All repos →