// the find
ConduitIO/conduit
Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
Conduit is a data pipeline tool written in Go that positions itself as a Kafka Connect replacement without the JVM tax. It runs connectors as external processes communicating over gRPC, meaning connectors can be written in any language. The target user is a team that needs CDC or ETL pipelines and is tired of babysitting a JVM cluster.
The goroutine-per-node pipeline model with Go channels is genuinely well-suited to this problem — you get real concurrency with backpressure for free, and the acknowledgement propagation (only acking upstream when all downstream destinations confirm) gets exactly-once semantics right. The gRPC connector protocol means the plugin boundary is well-defined and language-agnostic, which is a real architectural win over Kafka Connect's classloader hell. Configuration layering (CLI > env > YAML) with a consistent naming convention is unusually clean for a tool in this space. Having both a JavaScript processor escape hatch and a dead-letter queue out of the box covers the 90% case without requiring custom connectors.
601 stars for a project that's been around since 2022 is a sign the connector ecosystem is thin — the built-in list is File, Kafka, Postgres, S3, and generator/log for testing, which barely covers a fraction of what Kafka Connect's 200+ connectors do. The Kafka Connect wrapper exists but running a JVM process to wrap a JVM connector defeats half the pitch. There's a `pkg/lifecycle-poc` directory sitting in the production codebase alongside the real `pkg/lifecycle` — that's either unfinished work or an abandoned experiment that wasn't cleaned up, and it's a signal about internal discipline. BadgerDB as the default state store is a surprise choice: it's an embedded key-value store that will bite you on operational simplicity if you ever need to inspect or migrate pipeline state.