// the find
reugn/go-streams
A lightweight stream processing library for Go
go-streams is a Go library for building data pipelines using a Source → Flow → Sink model. No external dependencies in the core module; connectors for Kafka, NATS, Redis, Pulsar, and cloud storage live in separate submodules. Aimed at Go developers who want something lighter than Flink or Kafka Streams without dropping down to raw goroutines.
The core module is genuinely dependency-free, which matters if you're dropping this into an existing service. The windowing support (sliding, tumbling, session) is real and tested — most lightweight pipeline libraries skip this entirely. AdaptiveThrottler that backs off based on actual CPU/memory usage is a useful addition that you'd otherwise have to wire up yourself. Connector isolation via separate go.mod files means you don't pay the dependency cost of Kafka if you're only using Redis.
No backpressure model worth speaking of — channels are the buffer and when they fill you block, full stop. There's no concept of error handling lanes or dead-letter routing; a panicking element takes down the pipeline. The Keyed flow for parallel processing of grouped data has very little documentation on ordering guarantees or state management, which matters the moment you need anything beyond embarrassingly parallel workloads. The multi-module layout (each connector is its own go.mod) is correct in principle but makes version pinning across connectors annoying in practice.