Viewing profile — stkbailey

We didn't look into a dataflow engine specifically, in part because we have a heterogeneous set of workfloads. Our core use case is loading mission critical data in chunks, but it …

Interesting, Astronomer was actually my last choice for orchestrator. We went with Dagster, but I didn't want to make the takeaway "Dagster solves these problems", because it doesn…

Nope -- just MWAA, Astronomer, Dagster Cloud, and Prefect Cloud. In the past I used Argo Workflows pretty extensively and have talked about its pros and cons here: https://www.yout…

Yep, that's what I tried to point out in the article.

Author here - appreciate the comments and reads. To add a bit of color -- I spent about a month looking into orchestrators to migrate Whatnot's data platform onto earlier this year…