finds.dev

teamclairvoyant/airflow-maintenance-dags

A series of DAGs/Workflows to help maintain the operation of Airflow

View on GitHub

★ 1,770PythonApache-2.0updated Jun 2024

// the take

A collection of maintenance DAGs for Apache Airflow that handle the operational housekeeping most teams end up writing themselves: DB cleanup, log rotation, zombie task killing, and broken DAG pruning. If you run Airflow in production, you either have something like this already or your metastore is quietly growing without bound.

Drop-in DAGs with no framework dependency — copy the file, configure a few variables at the top, schedule it. The db-cleanup DAG is the most valuable piece; it handles DagRun, TaskInstance, XCom, and Log tables in one pass with configurable retention. The kill-halted-tasks DAG fills a real gap Airflow itself doesn't handle well — UI-killed tasks that keep running on executors. Each DAG is self-contained with its own README, so you can adopt one without touching the others.

Last commit was June 2024 and most DAGs appear to target Airflow 1.x/2.x APIs — there's no indication these work correctly on Airflow 2.8+ or 3.x, which has breaking changes in task execution and the metastore schema. No tests anywhere in the repo, so you're flying blind when you upgrade Airflow versions. The log-cleanup DAG uses SSH to clean worker nodes, which assumes a Celery executor setup and won't work with KubernetesExecutor or modern remote logging configs. The db-cleanup approach of directly querying ORM models is fragile against schema changes between Airflow versions.

We read the README, the directory tree and a key source file in June 2026. Nobody pays to be listed. How we judge →

// at a glance

Stars
1,770
Forks
396
Language
Python
License
Apache-2.0
Last commit
18 Jun 2024

// nearby

If airflow-maintenance-dags isn't quite it.

All repos →