Awesome Dagster
A collection of all things awesome related to the Dagster ecosystem!
Learning Material
- Dagster University: Dagster Essentials - A course for learning the basics of Dagster
- Dagster University: Dagster & dbt - A course on how to use Dagster with dbt
- Dagster Open Platform - A full-sized Dagster project that contains real assets that are used by the Dagster Labs team
- Dagster Labs: DevRel Project Demos - Example projects shared by the Developer Relations team at Dagster
- A Dagster Crash Course - A crash course of Dagster from Pete Hunt
- Data Engineering Vault - Data engineering concepts related to Dagster (see graph and backlinks).
- Dagster DevRel Project Demos - A collection of projects and presentations from Dagster's devrel team
- A History of Data Orchestration - From bash to stored procedures to ETL to Python scripts including Dagster.
Articles
- The Dagster Blog
- Building a Data Engineering Project in 20 Minutes - September 3, 2021
- Dagster 1.0 Launch Recap - Simon Späti | August 11, 2022
- The Open Data Stack Distilled into Four Core Tools - Simon Späti | January 3, 2023
- Complete Guide To Deploy Multiple Data Pipelines to Dagster on Docker Environment - Ibrahim Halil Koyuncu | September 21, 2023
- Dagster, dbt, duckdb as new local MDS - Aleksandar Milicevic, Georg Heiler | December 11, 2023
- Complete Guide To Deploy Multiple Data Pipelines on Kubernetes Environment - Ibrahim Halil Koyuncu | December 17, 2023
- “How to Fine-Tune LLMs in 2024 with Hugging Face”, but with Dagster, Modal and Llama3! - Kyryl Truskovskyi | April 21, 2024
- Complete Guide To Deploy Multiple Data Pipelines on AWS ECS - Ibrahim Halil Koyuncu | April 24, 2024
- Cost efficient alternative to databricks lock-in - Georg Heiler, Hernan Picatto | May 3, 2024
- RLHF with Dagster and Modal - Kyryl Truskovskyi | June 10, 2024
- Building a po' boy's spatial data pipeline from scratch with Dagster, GeoParquet, & R2 - Alex Lowell Martin | July 16, 2024
- Data Stack in a Box — New South Wales Department of Education (ft. DuckDB, Dagster, dbt, dlt and Evidence.dev) - David Griffiths | Jul 21, 2024
- Remove Complexity from Your RAG Applications (Modal, LanceDB, and Dagster) - Kyryl Truskovskyi | Sep 9, 2024
- Orchestrating an AI Agent Prototype with Hex, Dagster, and LangChain - Olivier Dupuis | Oct 6, 2024
Talks
- Data Engineering Podcast - Build Maintainable And Testable Data Applications With Dagster - Episode 104 October 28, 2019: Fist podcast about Dagster where Nick Schrock the founder explains the motivation and the story behind Dagster.
- Software Engineering Daily - Dagster with Nick Schrock November 15, 2019: Second podcast with different insight from the software engineering perspective
- Python Podcast - Digging Into Dagster: An Opinionated Open Source Framework For Data Orchestration - Episode 279 September 7, 2020: Update on Dagster after a year.
- Big Data Madison
- CNCF - Ditching Data Pipelines: Why Treating Data as Assets is the Best - Andrea Giardini
Projects
- Coches.net Dashboard
- Practical Data Engineering Project
- Portable Data Stack
- Dagster Graph Project
- Dagster Open Platform
- MIT Open Learning
- Catalyst Cooperative PUDL
- Datadex
- Anomstack
- dagster-nomad Nomad launcher/executor for Dagster
- Dagster Demo - A working dagster demo with medallion architecture, partitioned data, schedules, assets dependencies, job status alerts and auto-materialize assets.
- Dagster, dbt, duckdb - Dagster, dbt, duckb as the new local MDS
- Dagster Utilities - Basic utilities for handling production vs local Dagster instances, importing jobs and schedules into your Definitions object, and more.
- Dagster + Mflix ELT Project - A working demo of Dagster, dlt and Snowflake for ELT
- Bike Rental Data Pipeline with Weather Data
- Orchestrating ML Workloads with Modal and Dagster
- Filecoin Data Portal - Running Dagster and dbt inside a Github Action
- Pipes for PostgreSQL UDF
- NCAA Basketball Stats Dagster Project and Supplementary video
Discussion
Community
- Dagster Slack - Join the community and Dagster Labs team members on Slack
- Dagster Github Discussion - Discuss features and contributions on GitHub
Company Projects
- Bestplace - Building a Modern Data Analytics Platform with Dagster
- Doordash - Increasing Operational Efficiency with Scalable Forecasting
- Drizly - Building shared spaces for data teams at Drizly
- Geomagical Labs (Ikea) - Building Customer-Facing Applications with Dagster
- GoodEggs - Good Data at Good Eggs
- LINE (Thailand) - Build Data Application with Dagster
- MIT Open Learning - Running Dagster in Production
- Mapbox - Incrementally Adopting Dagster at Mapbox
- Prezi - Migrating Data Pipelines to Dagster
- VMware - Monitoring Driven Development with Dagster
For more awesome projects see the community meetings Community Member Showcase!