// the find
awslabs/data-on-eks
DoEKS is a tool to build, deploy and scale Data Platforms on Amazon EKS
A collection of Terraform blueprints and example manifests for running data workloads (Spark, Flink, Kafka, Ray, Airflow, ClickHouse, Trino) on Amazon EKS. Maintained by AWS solution architects, it targets data engineers who want opinionated starting points rather than blank-slate Kubernetes configs. Not a product — a reference library.
The breadth of coverage is real and useful: each data stack gets its own deploy/cleanup scripts, Terraform vars, and working example manifests, so you can actually run a Spark job or stand up Strimzi Kafka without inventing all the glue yourself. The benchmark artifacts are a standout — TPC-DS results comparing Spark with Celeborn, Gluten/Velox, DataFusion Comet, and GPU RAPIDS give you actual numbers to reference when choosing a shuffle strategy. Karpenter integration is baked into most stacks, which is the right call for cost management on EKS. The CI includes parallel e2e plan runs and a daily tag workflow, so the Terraform isn't completely bit-rotted.
Everything is EKS-specific to the point of being AWS-locked by design — if you're on GKE or self-managed Kubernetes, nothing here transfers. The blueprints are also one-size-fits-one: the Terraform tfvars files are opinionated defaults with no documented tuning guide for production workloads, so you're on your own figuring out what to change for your data volume or latency requirements. At 854 stars for a repo covering this many frameworks, community momentum is thin — many stacks have sparse issue history, meaning if you hit a bug in the Pinot or DataHub blueprints you're mostly debugging alone. The 'in active development' caveat in the README has been there long enough that it reads more like a liability disclaimer than a status update.