// the find
estebanpdl/osintgpt
An open-source intelligence (OSINT) analysis tool leveraging GPT-powered embeddings and vector search engines for efficient data processing
osintgpt is a Python CLI and library that indexes a folder of documents you already hold and answers questions over them, with citations back to the source passages. It suits analysts working through a fixed research collection who want local or hosted models and a store they control, not anyone who needs to collect new material from the open web.
- Storage and model providers are separate, pluggable settings. SQLite is the default store, Qdrant and Postgres are supported, and embeddings can run through sentence-transformers or Ollama with no code changes, so the fully local path is configuration rather than a fork.
- Indexing is checkpointed. Successful embedding batches are journaled in index-journal.sqlite and reused on rerun, and the index fingerprint covers the field mapping, chunking rules, model and destination, so a config change rebuilds only the affected files. Most tools skip this and regret it once a provider rate-limits a 50,000-file corpus halfway through.
- Graph edges keep their source document and the quoted evidence, and osintgpt graph verify rechecks those quotes against the text. That gives you a way to audit extracted relationships before repeating them, which matters more in this domain than the usual RAG demo.
- The OSINT label oversells it. Nothing here collects anything. It reads files you add, so anyone expecting scraping, social-source collection or monitoring will be disappointed.
- The README runs long, and a large share of it documents retry budgets, RPM/TPM ledgers and quota scopes. That is a lot of operational machinery for a 0.3.0 release, and it is the part most likely to surprise someone configuring it.
- The canon directory is documented as a folder and link structure only. The README says osintgpt does not populate it, so the Obsidian angle is a plan rather than a feature.
- The fully local path is heavy. The [local] extra pulls in torch and sentence-transformers, and generation needs a separately installed Ollama, so the default pip install is light but a private setup is a large download and a second service to run.