// the find
couler-proj/couler
Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.
Couler is a Python abstraction layer that lets you write workflow DAGs once and (theoretically) run them on Argo Workflows, Airflow, or Tekton. In practice it's Argo-only today, with Airflow support pegged at 40-50% complete. Aimed at ML platform teams who don't want to rewrite pipelines every time the orchestration layer changes.
The Python DSL is genuinely cleaner than raw Argo YAML or Kubeflow's class-heavy API — the coin-flip and DAG examples show real ergonomic improvement. The IR/proto design is architecturally sound for multi-engine support even if that promise isn't fully delivered yet. It ships with first-class ML-specific step types (TensorFlow, PyTorch, MPI, Katib) baked in, not bolted on. There's an academic paper (ICDE 2024) and a reported 3000+ user deployment at Ant Group, so the core Argo path has real production mileage.
The multi-engine pitch is mostly vaporware right now — the README openly admits only Argo is fully supported, and the Tekton design doc in /docs appears to be aspirational rather than implemented. Last push was October 2024 and activity has been sparse, which is a red flag for a project that depends on keeping up with rapidly-changing upstream engines. The install story is rough: no PyPI release, you're pip-installing directly from git. The global mutable state pattern in couler/core/states.py means you can't safely compose multiple workflows in one process without manual cleanup, which will bite anyone building a dynamic workflow generator.