// the find
cube2222/octosql
OctoSQL is a query tool that allows you to join, analyse and transform data from multiple databases and file formats using SQL.
OctoSQL is a CLI tool that lets you run SQL queries across heterogeneous data sources — join a CSV against a Postgres table, query a JSON file, pipe from stdin — all in one command. It's a full dataflow engine under the hood, not a thin SQLite loader, with static typing, predicate pushdown, and streaming watermark support. The target user is a developer or data engineer who lives in the terminal and needs ad-hoc cross-source analysis without spinning up a pipeline.
The streaming and watermark model is genuinely engineered, not bolted on — internal consistency guarantees, retraction-based updates, and tumbling window support put it closer to a mini Flink than a CSV-to-SQL shim. The union type system handles messy real-world data (a column that's sometimes Float, sometimes String) without forcing you to cast everything upfront. Predicate pushdown into lookup joins is real and visible via --explain, which matters when the right side is a million-row Postgres table. Performance on CSV queries is competitive with non-cached q and textql while running directly on the file without a SQLite intermediate step.
The repo explicitly does not accept external contributions to source code, only plugins — which is an unusual choice for an open-source project at this stage and will slow community-driven datasource expansion. The plugin ecosystem is thin: the core repository has only a handful of entries, so you'll hit a wall quickly if your datasource isn't CSV, JSON, Parquet, Postgres, or MySQL. Last push was May 2024 and activity looks sparse, which is a real adoption risk for anything you'd depend on. DataFusion beats it by 4.5x on the benchmark used, and for pure file-query workloads that gap matters.