// the find
nisshi-io/nisshi
Apache Kafka® compatible broker with S3, PostgreSQL, SQLite, Apache Iceberg and Delta Lake
Nisshi (formerly Tansu) is a Kafka-protocol-compatible broker written in Rust that swaps ZooKeeper/KRaft and local disk for pluggable storage — PostgreSQL, SQLite, S3, or memory — and can write schema-validated topics straight out to Iceberg or Delta Lake tables. It's aimed at teams who want Kafka's wire protocol without running Kafka's operational stack, or who want streaming data landing directly in a lakehouse without a separate Connect/CDC hop.
The storage engine abstraction is real, not cosmetic — memory for local dev, Postgres or S3 for durability, with S3's eleven-nines pitched as the production story. Writing Avro/JSON/Protobuf-validated topics directly to Iceberg or Delta Lake collapses a pipeline that normally needs Kafka Connect or Debezium into one binary. Compatibility isn't just claimed: there's a compat/ directory running the broker against real librdkafka and franz-go client test suites with recorded FINDINGS.md, plus fuzz targets for the wire protocol's varint and batch decoding.
TLS was accepted as a flag before 0.7 but silently did nothing — anyone who deployed with --cert/--key thinking they had encryption was serving plaintext the whole time, which is the kind of bug that should worry you about what else looked configured but wasn't. Even now, the bundled cat/topic/proxy/perf CLIs can't speak TLS, so a secured broker deployment breaks its own tooling. Client auth is SASL riding on top of TLS's encryption-only layer, and the README doesn't spell out which SASL mechanisms are actually supported. More fundamentally, this is a from-scratch reimplementation of Kafka's broker semantics — consumer group rebalancing, transactions, offset management — and getting those bit-for-bit compatible is notoriously hard; worth stress-testing rebalance and transactional-produce paths yourself before trusting it over real Kafka or Redpanda in anything that matters.