Curated resources on pipeline orchestration: DAG concepts, Airflow, Dagster, and choosing the right scheduler for your data stack.
The authoritative source on how Airflow models dependencies, scheduling, and task execution. Read this before any tutorial.
Dagster's asset-centric model is a conceptual upgrade over Airflow's task-centric model. The getting-started tutorial is one of the best in the data tooling space.
The definitive guide to writing production-quality Airflow DAGs. Covers idempotency, dynamic task mapping, and the common pitfalls that cause silent failures.
Prefect is the third major orchestrator worth knowing. Its hybrid execution model (cloud orchestration, local workers) fits teams that cannot run their own Airflow cluster.
Some course links above are affiliate links. If you enroll, we may earn a small commission at no extra cost to you.
New resources and perspective on building AI-ready data systems, a few times a month. No spam.