// the find
mage-ai/mage-ai
🧙 Build, run, and manage data pipelines for integrating and transforming data.
Mage is a self-hosted data pipeline tool with a notebook-style block editor — think Jupyter meets Airflow, but with a visual DAG UI and built-in connectors for the usual suspects (Snowflake, BigQuery, Postgres, S3, Kafka, etc.). It targets data engineers who want something less ceremonial than Airflow for local development and simple production jobs. The OSS version is openly a feeder for Mage Pro, with the serious production features (RBAC, multi-env orchestration, monitoring, CI/CD) gated behind the paid tier.
- The connector catalog is genuinely wide — 50+ sources and destinations documented, including less-common ones like Chargebee, Outreach, and Teradata, which saves real integration grunt work.
- dbt is a first-class citizen: you can run dbt models inside a Mage pipeline block without stitching together a separate orchestrator, which is a meaningful DX win for teams already on dbt.
- Block-level execution and live data previews make iteration fast — you can run a single transformer block and inspect the output without running the whole pipeline, which Airflow cannot do.
- Docker setup is genuinely one command; the pip and conda paths also work, which is more than most tools of this scope can claim.
- The OSS/Pro split is aggressive. RBAC, real monitoring and alerting, multi-environment support, and CI/CD integration are all Pro-only. If you're running this for a team beyond a solo analyst, you'll hit the paywall fast.
- Notebook-style pipeline authorship creates version control pain — pipeline state is stored in YAML metadata alongside Python files, and merge conflicts in that structure are unpleasant to resolve.
- No serious competitor to Airflow's provider ecosystem for operators. If you need to wait on an SQS message, trigger a Databricks job, or interact with anything niche, you're writing a custom block from scratch.
- The self-hosted story for production is underspecified in OSS — there's no built-in HA, no worker autoscaling, and the docs for running this reliably on Kubernetes outside of Mage Pro are sparse.