// the find
apache/cassandra-spark-connector
Apache Spark to Apache Cassandra connector
The official Apache Spark connector for Cassandra, originally built by DataStax and donated to the ASF. It lets you read Cassandra tables as Spark DataFrames/RDDs and write results back, with server-side predicate pushdown and partition-aware shuffling. It's the only serious option if you're running Spark + Cassandra together.
The `joinWithCassandraTable` RDD operation is genuinely useful — it fetches only the Cassandra rows you need based on Spark partition keys, avoiding a full table scan. Partition-aware repartitioning (`repartitionByCassandraReplica`) can cut shuffle overhead significantly for co-located processing. Cassandra 5.0 vector type support landed in 3.5.1, so it now works with the new SAI vector indexes for ANN queries. The integration test suite is thorough and runs against real Cassandra instances via CCM, not mocked drivers.
Version pinning is painful — every Spark minor version needs its own connector branch, so if you upgrade Spark 3.4 → 3.5 you need a connector upgrade too, and the cadence has historically lagged. Scala 2.11 support is fully dead and 2.12 is deprecated in 3.5.x, which will break anyone still on older Spark clusters. The CCM dependency for integration tests means local dev setup is non-trivial — you need a full Cassandra cluster manager installed, not just Docker. Last push was April 2025 with no Spark 4.x branch visible yet, which is going to become a problem soon.