// the find
lw-lin/CoolplaySpark
酷玩 Spark: Spark 源代码解析、Spark 类库等
A Chinese-language deep-dive into Spark internals, covering both the legacy Spark Streaming (1.x) and Structured Streaming (2.x) source code. It's a documentation/analysis repo, not a library — the value is the annotated walkthroughs of how Spark's streaming engine actually works under the hood. Target audience is Spark practitioners who read Chinese and want to understand why things behave the way they do, not just how to use the API.
The coverage of Structured Streaming internals is genuinely thorough — sources, sinks, state store, event time, and watermark each get their own dedicated analysis with diagrams. The separation between Spark Streaming (1.x micro-batch) and Structured Streaming (2.x) is clear, which matters because the two are architecturally quite different. The diagrams throughout the streaming analysis series are hand-drawn and actually explain the data flow rather than just restating the API docs. Whoever wrote this clearly read the Spark source, not just the user guide.
Last updated 2022 against Spark 2.x — Spark has since shipped 3.x with substantial changes to the streaming internals, so some of this is now historical documentation rather than current. The repo is entirely in Chinese, which cuts out most of the global developer community and limits its reach for anyone who doesn't read it. There's basically no runnable code beyond a hello-world Maven skeleton; it's a reading guide, not something you can run against your own cluster. Coverage stops abruptly — the Structured Streaming series ends at watermark, skipping joins, arbitrary stateful processing, and continuous processing mode.