Netflix has open-sourced its Maestro Workflow Orchestrator
1–10 of 42 posts
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#2This appears to be the high level architecture: https://miro.medium.com/v2/resize:fit:4800/format:webp/0*SDt....
To me seems to overengineered for most of the companies outside there, how many people do you need to manage it?
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#3I was hoping to try this out, but it doesn't build on my mac :/
Here's their tech blog post about it: https://netflixtechblog.com/orchestrating-data-ml-workflows-...
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#4I was hoping to try this out, but it doesn't build on my mac :/ Here's their tech blog post about it: https://netflixtechblog.com/orchestrating-data-ml-workflows-...
[flagged]
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#5I was just thinking didn't Netflix already have its own workflow engine: https://github.com/Netflix/conductor, but found out it's archived/EOL so this makes more sense now.
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#6This appears to be the high level architecture: https://miro.medium.com/v2/resize:fit:4800/format:webp/0*SDt... . To me seems to overengineered for most of the companies outside there, how many people do you need to manage it?
In the blog, it claims:
> … schedule hundreds of thousands of workflows, millions of jobs every day and operate with a strict SLO of less than 1 minute of scheduler introduced delay even when there are spikes in the traffic.
I wouldn’t necessarily call it over engineered if it can handle this type of workload.
But if you are running a fraction of these workloads with a much more relaxed SLO (service level objection), then yea this is overengineered.
Like any solution from FAANG, or “big tech”. It works for _them_ at scale but for 90% of companies the sheer management of this service will quickly outweigh the benefits.
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#7[deleted]
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#8After almost demise of nomad, market needs another general purpose orchestrator that isn't Kubernetes or docker swarm.
This is not it however.
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#9I was just thinking didn't Netflix already have its own workflow engine: https://github.com/Netflix/conductor , but found out it's archived/EOL so this makes more sense now.
[deleted]
Re: Netflix has open-sourced its Maestro Workflow Orchestrator
#10I was just thinking didn't Netflix already have its own workflow engine: https://github.com/Netflix/conductor , but found out it's archived/EOL so this makes more sense now.
Conductor was moved here: https://github.com/conductor-oss/conductor