// the find
supabase/etl
Stream your Postgres data anywhere in real-time. Simple Rust building blocks for change data capture (CDC) pipelines.
supabase/etl is a Rust library and binary for Postgres logical replication pipelines. It handles the initial table copy, ongoing CDC streaming, and state recovery in a single process — no Kafka or Debezium required. Aimed at teams who want to replicate Postgres data to analytics destinations (BigQuery, DuckLake) without standing up a separate streaming infrastructure.
The single-process design is a real differentiator — most CDC setups require a coordination layer that doubles your ops burden. The typed Rust API means schema mismatches blow up at compile time rather than silently corrupting your data warehouse. Memory backpressure config (`memory_budget_ratio`, `max_bytes`) shows they've thought about production failure modes, not just happy-path throughput. Test coverage is solid: snapshot tests for K8s manifests, integration tests with real DB support, and a benchmark crate for regression tracking.
Not on crates.io yet — git dependencies in production are a supply-chain headache and make version pinning fragile. The destination story is thin: BigQuery is 'stable', DuckLake is 'in progress', and Iceberg was deprecated before it shipped, which doesn't inspire confidence in the project's direction. Schema change support is documented as 'simple DDL changes today' with destination-specific behavior — in practice this means ALTER TABLE on a busy table will likely break your pipeline in ways that aren't obvious until after the fact. The REST API crate (`etl-api`) is clearly internal Supabase infrastructure but ships in the same repo with no stability guarantees, which will confuse adopters trying to understand what surface is actually public.