I was just thinking didn't Netflix already have its own workflow engine: https://github.com/Netflix/conductor , but found out it's archived/EOL so this makes more sense now.
Conductor was moved here: https://github.com/conductor-oss/conductor
Netflix has open-sourced its Maestro Workflow Orchestrator
31–40 of 42 posts
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#32This appears to be the high level architecture: https://miro.medium.com/v2/resize:fit:4800/format:webp/0*SDt... . To me seems to overengineered for most of the companies outside there, how many people do you need to manage it?
That chart has real Krazam Microservices vibes to it.
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#33This appears to be the high level architecture: https://miro.medium.com/v2/resize:fit:4800/format:webp/0*SDt... . To me seems to overengineered for most of the companies outside there, how many people do you need to manage it?
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#34Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#351) Scheduling seems to be primitive? (strategy: sequential) 2) That's seems to be just a DAG executor? No high-level frontend? 3) No executiun context? 4) No typings? 5) No concept of a stream? It would be way too ambitious to call this thing an orchestrator, it seems to be just a primitive distributed DAG executor.
It's hard to take your comment seriously when this was one of the earliest talking points.
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#36Earlier quoted context omitted.
That chart has real Krazam Microservices vibes to it.
Here's a video of Primeagen, a YouTuber who uses to be a dev at Netflix, watching that video and then telling a very similar story that happened at Netflix: https://youtu.be/s-vJcOfrvi0
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#37Earlier quoted context omitted.
It doesn't build on windows either. It looks to be missing some dependency. > Did not resolve 'com.github.vmg.protogen:protogen-annotations:1.0.0' which is part of the dependency lock state
I get a similar error
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#38I was hoping to try this out, but it doesn't build on my mac :/ Here's their tech blog post about it: https://netflixtechblog.com/orchestrating-data-ml-workflows-...
./run.sh maestro
[1] https://github.com/data-catering/insta-infraRe: Netflix has open-sourced its Maestro Workflow Orchestrator
#391) Scheduling seems to be primitive? (strategy: sequential) 2) That's seems to be just a DAG executor? No high-level frontend? 3) No executiun context? 4) No typings? 5) No concept of a stream? It would be way too ambitious to call this thing an orchestrator, it seems to be just a primitive distributed DAG executor.
Did you miss the sections on how this allows cyclic graphs and not just DAG? It's hard to take your comment seriously when this was one of the earliest talking points.
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#40Earlier quoted context omitted.
Did you miss the sections on how this allows cyclic graphs and not just DAG? It's hard to take your comment seriously when this was one of the earliest talking points.
> In Maestro, a workflow is a DAG (Directed acyclic graph) of individual units of job definition called Steps.
> Unlike traditional workflow orchestrators that only support Directed Acyclic Graphs (DAGs), Maestro supports both acyclic and cyclic workflows and also includes multiple reusable patterns, including foreach loops, subworkflow, and conditional branch, etc.
https://netflixtechblog.com/maestro-netflixs-workflow-orches...