// the find
deepflowio/deepflow
eBPF Observability - Distributed Tracing and Profiling
DeepFlow is a zero-instrumentation observability platform using eBPF to automatically collect metrics, distributed traces, and continuous profiling data from Kubernetes and cloud-native environments. The agent is written in Rust, the server in Go, and it backs onto ClickHouse with a custom 'SmartEncoding' tag compression scheme. Target audience is SRE/platform teams running complex microservice environments who want observability without touching application code.
- Agent written in Rust with genuine eBPF-based network capture, DWARF unwinding for continuous profiling, and pcap-backed protocol parsers tested against real packet captures — this is non-trivial systems work, not a thin wrapper.
- SmartEncoding stores tags as pre-encoded integers rather than strings in ClickHouse, which is a real engineering solution to the cardinality/storage problem that plagues most OTel-based stacks.
- Protocol coverage is wide and properly tested: DNS, HTTP/1/2, gRPC, Kafka, Dubbo, Redis, MySQL, AMQP — each with pcap golden files, not just unit tests against mocked data.
- SIGCOMM 2023 paper acceptance for the network-centric tracing approach gives the core algorithm more credibility than most observability projects can claim.
- The open-source community edition is clearly a stripped-down version of the commercial Enterprise product — several plugin stubs (npb_handler, npb_sender, packet_dedup) are empty shells, and the enterprise feature split is not clearly documented, so you may hit walls mid-adoption.
- Deployment complexity is real: requires a K8s cluster for the server, ClickHouse, plus the agent DaemonSet; the 'all-in-one' quick start docs are hosted externally and the repo itself gives you almost nothing to bootstrap locally without reading through external docs.
- Wasm plugin API for custom protocol parsers is mentioned prominently but documentation for actually writing one is sparse in the repo itself — the feature is real but the developer experience for extending it is unclear.
- Heavy Chinese-first development culture: WeChat group as primary community channel, Enterprise demo is Chinese-only, and commit/issue activity suggests the English community is secondary, which matters for long-term support expectations.