Dagster

Today we're launching Dagster+, the next evolution of Dagster Cloud. Dagster+ is targeted at data engineers, ML engineers, and analytics engineers, and is a major rethinking of what a data orchestrator is, where it sits in the stack, and how data platforms are constructed. Since our launch in August 2022, we have seen enormous growth in both the Dagster Core open-source project and our commercial product. We now have hundreds of commercial customers and the fastest-growing open-source community in the category. With this growth comes broader and deeper relationships with a diverse set of customers. As we engaged with these customers throughout 2022 and 2023, we heard a few key challenges: • They were spending too much money on infrastructure, especially on the data warehouse, ETL tools, and AI workloads, and struggled to understand, optimize, and control their spend. • Their stacks were sprawling and complex, with hundreds of different integrations and point solutions with varying levels of maintenance, and no single pane of glass for understanding the data platform as a whole. • They wanted to empower individual teams to work autonomously end-to-end, but struggled to balance this with the centralized needs of the data organization. Dagster+ is a suite of tools that sits on top of Dagster's data orchestration engine, and replaces the existing Dagster Cloud product. Dagster+ brings four new capabilities above and beyond our existing commercial product: 1️⃣ We have built a brand new data catalog and data discovery capability in Dagster+, which enables all users of the data platform to discover and leverage data autonomously. 2️⃣ Dagster+ builds on Dagster Core’s asset checks feature, introducing additional capabilities around anomaly detection, freshness, and alerting. 3️⃣ We’ve made massive improvements to our Branch Deployments feature from Dagster Cloud, and are launching a killer new feature, Change Tracking, with Dagster+. 4️⃣ Dagster Insights enables data teams to manage their spend on tools like Snowflake, Fivetran and OpenAI without doing any additional integration work. With Dagster+, Insights is now generally available and comes with many new features and integrations. All of these capabilities are enabled by Dagster's unique asset-oriented approach to data orchestration. I'm really proud of the product we've built, and I'm proud to be a part of the team that delivered it. We hope that Dagster+ helps you reduce your spend, eliminates the need for many point solutions, and empowers your teams to work more autonomously and less chaotically.

Congrats on the launch of Dagster+! 🎉 It’s fantastic to see how you’ve listened to the community and developed new featuers that address the real challenges faced by data teams.The introduction of the data catlog, change tracking, and enhanced spend managment tools - shows a deep understanding of what’s needed to simplify and streamline data operations. Really excited to see how Dagster+ will transform data orchestration and empower teams to work more efficiently and autonomously. Cheers to a future with less chaos and more inovation in data handling! 🚀

Congratulations on the launch! I'm particularly intrigued by the data catalog and discovery capability, as well as the improvements in spend management with Dagster Insights. How do you envision these new features transforming the data platform landscape for your customers?