// the find
karafka/karafka
Ruby and Rails efficient Kafka processing framework
Karafka is a multi-threaded Kafka consumer framework for Ruby and Rails, built on top of librdkafka via the rdkafka gem. It handles the plumbing of partition assignment, offset management, and consumer group coordination so you write consumer classes instead of Kafka boilerplate. Target audience is Rails shops that want to process Kafka events without dropping down to raw rdkafka or building their own threading model.
Built on librdkafka, so you get a battle-tested C client underneath with proper rebalance handling and at-least-once semantics. The strategy combinatorial pattern (dlq_ftr_lrj_mom_vp etc.) is ugly to look at in the tree but actually sound — it makes feature interaction explicit rather than leaving it to runtime surprises. ActiveJob backend integration is real and not an afterthought; ordered jobs and DLQ support on top of it saves Rails teams significant custom work. The Web UI ships as part of the ecosystem, not a separate commercial product, and the Kubernetes liveness listeners suggest production deployments are a first-class concern.
The Pro license split is more aggressive than most OSS projects — virtual partitions, long-running jobs, and connection multiplexing are all behind a commercial paywall, so you may prototype on OSS and then find the features you actually need are Pro-only. The strategy file explosion (50+ files for feature combinations) indicates the architecture doesn't compose cleanly; adding a new feature means touching every combination matrix, which will bite contributors. No native schema registry or Avro/Protobuf support in the core — the examples directory has a nearly empty avro folder, which is a signal that this is left entirely to the user. Being single-maintainer (karafka is essentially one person's project) is a real bus-factor risk for a framework sitting in your critical message-processing path.