Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nomad, a cluster manager and scheduler

hashicorp.com

Post reply on HN
Print view View original

31–40 of 48 posts

Re: Nomad, a cluster manager and scheduler

#31

Post by cpitman »

amouat wrote: post #27
cpitman wrote: post #25

This one is interesting to me. I've been a big proponent of Hashicorp's other tooling, but this seems like an area that other projects are already addressing (and doing well in). Choice is great, but I think I would have preferred if they joined up with Kubernetes/Mesos/etc. Also, their messaging seems a little ingenious. Otto talks about how important it is to support microservice development and deployment, but Nom…

Did you mean ingenious? Or disingenuous?

Ha, you are correct, I meant disingenuous.

Re: Nomad, a cluster manager and scheduler

#32

Post by illamint »

rgarcia wrote: post #30
cpitman wrote: post #25

This one is interesting to me. I've been a big proponent of Hashicorp's other tooling, but this seems like an area that other projects are already addressing (and doing well in). Choice is great, but I think I would have preferred if they joined up with Kubernetes/Mesos/etc. Also, their messaging seems a little ingenious. Otto talks about how important it is to support microservice development and deployment, but Nom…

Also, their messaging seems a little disingenuous. Otto talks about how important it is to support microservice development and deployment, but Nomad lists as a con that Kubernetes has too many separately deployed and composed services. This is consistent with a (reasonable) belief that microservice architecture is an important design pattern to support, but may not be the best approach for all problems. From reading…

I think there's definitely a bootstrapping problem here: microservices are great if you have something like Kubernetes, Nomad, Mesos etc. on which to run and deploy them, but you have to run your platform on something and be able to bring it back up if it goes down and that's where I think Nomad might have the edge.

Re: Nomad, a cluster manager and scheduler

#33

Post by SEJeff »

I'm genuinely not sure why they are trying to compete with the likes of Mesos or Kubernetes, or what they are really trying to achieve here. There is simply no way they'll build a community around nomad 1/2 as big as either of the aforementioned even if the software is really good.

Re: Nomad, a cluster manager and scheduler

#34

Post by larryweya »

I've been using Mesos with Aurora in production for almost a year and while its very stable (zero downtime so far with loss of hosts a couple of times) the setup process was quite tedious and not something I'm looking forward to doing again. It also has a lot of moving parts - understand and setup zookeeper to get mesos up, understand and setup Aurora, use something for service discovery (I use AirBnB's Synapse for this). Plus I prefer to use tools I can choose to look under the hood of and perhaps make some contributions, which is a bit intimidating with both Mesos and Aurora (C/C++ and Scala). Because of this, I'm keen to try out Nomad mostly because of the promise of no-other-dependency/single binary plus the use of a single language across the stack - Golang

Re: Nomad, a cluster manager and scheduler

#35

Post by stevelandiss »

pm90 wrote: post #8

> Nomad is designed to be a global state, optimistically concurrent scheduler. Global state means schedulers get access to the entire state of the cluster when making decisions enabling richer constraints, job priorities, resource preemption, and faster placements. Can anyone shed light on how that's possible? I was under the impression that global state in distributed systems was not possible?

Why would you say that global state in a distributed system is not possible? The entire internet is a distributed system with global state. The state needs to be aware of the context and access, that's all.

Re: Nomad, a cluster manager and scheduler

#36

Post by fapjacks »

SEJeff wrote: post #33

I'm genuinely not sure why they are trying to compete with the likes of Mesos or Kubernetes, or what they are really trying to achieve here. There is simply no way they'll build a community around nomad 1/2 as big as either of the aforementioned even if the software is really good.

I'm sure to a fan that "really freaking loves Mesos" you can't imagine other software competing with Mesos, but Hashicorp writes excellent software generally. Serf and Consul have completely sold me on Nomad, and I haven't even used it yet. I think there will be similar draw for many people that have used Hashicorp's software before.

Re: Nomad, a cluster manager and scheduler

#37

Post by SEJeff »

fapjacks wrote: post #36
SEJeff wrote: post #33

I'm genuinely not sure why they are trying to compete with the likes of Mesos or Kubernetes, or what they are really trying to achieve here. There is simply no way they'll build a community around nomad 1/2 as big as either of the aforementioned even if the software is really good.

I'm sure to a fan that "really freaking loves Mesos" you can't imagine other software competing with Mesos, but Hashicorp writes excellent software generally. Serf and Consul have completely sold me on Nomad, and I haven't even used it yet. I think there will be similar draw for many people that have used Hashicorp's software before.

No it isn't about koolaid, it is about technology. Consul, Vagrant, and Vault are all excellent Hashicorp technologies I've used. They are great, and some of the best in the industry for their problem spaces.

You've got Mesos scaling to 10,000+ physical node clusters today in production at the likes of companies like Apple and Twitter. You've got Kubernetes being adopted and developed by pretty much all of the open source heavyweights and it came from the experienced Google developed building... Google. Kubernetes ontop of Mesos is basically the holy grail in my personal opinion where you get the best Ops (mesos) story mixed with the best Dev (k8s) story. I guess we'll see how much Nomad takes off :)

Hashicorp isn't a huge company, it seems like to me their best bet is keeping the focus relatively small so they can be the best at what they do. Even if Nomad is a huge hit and is amazing, it still seems kind of sad that they couldn't simply double down and help out with kubernetes. I do find it ironic that they talk about how nomad is for microservices and then make a dig at the several microservices that k8s is made up of.

Re: Nomad, a cluster manager and scheduler

#38

Post by atombender »

SEJeff wrote: post #33

I'm genuinely not sure why they are trying to compete with the likes of Mesos or Kubernetes, or what they are really trying to achieve here. There is simply no way they'll build a community around nomad 1/2 as big as either of the aforementioned even if the software is really good.

This is a space where you can compete on operational complexity. Mesos and Kubernetes are fairly complicated to set up and run, with dependencies and intricacies that Nomad just doesn't have.

From your other comment:

> I do find it ironic that they talk about how nomad is for microservices and then make a dig at the several microservices that k8s is made up of.

Being for microservices doesn't mean you should be a microservice. Kubernetes your control plane involves 5 services (Kubelet, proxy, Docker, replication controller, etcd) that need to be up and running before you even have started an app. Then the question is what happens to your system if one or two of those go down, or if they become bottlenecks, or need to be upgraded, and so on.

Having evaluated both Mesos and Kubernetes, Nomad is a lot more attractive to me due to its simplicity and "turnkey" approach.

Re: Nomad, a cluster manager and scheduler

#39

Post by josh2600 »

derefr wrote: post #14
josh2600 wrote: post #9

Earlier quoted context omitted.

http://research.google.com/pubs/pub41684.html Basically, there's a server that holds state for the cluster. When a scheduler attempts to load a job into the cluster, it grabs state from the aforementioned server, performs its job placement calculations, and then tries to submit its answer to the master state. Since there are many such schedulers, and the amount of time it takes to place a job is non-trivial, its poss…

Sounds more like "greedy" than "optimistic" scheduling: the early jobs get the worms. In most scheduling systems, two 10,000-node jobs that each would saturate a cluster on their own will time-share if submitted together, with each job acting in practice more like 10,000 single-node jobs. The result is usually each job getting a probabilistic 50% share of the cluster while they're both running, and then whichever one…

This job swapping problem is a different sort of problem. In your example, I have cluster saturation as a concern. Google would love to saturate their cluster, but I would wager they usually are nowhere near saturation (scheduling a job on Borg takes something like 90 seconds at minimum; there's a lot of ground to cover). The innovation in optimistic schedulers is that you can have multiple schedulers running against a master state. This is in contrast to a single scheduler, single state system like Mesos (forgive me if I've misspoken about Mesos, but this is how I remember it).

Re: Nomad, a cluster manager and scheduler

#40

Post by SEJeff »

atombender wrote: post #38
SEJeff wrote: post #33

I'm genuinely not sure why they are trying to compete with the likes of Mesos or Kubernetes, or what they are really trying to achieve here. There is simply no way they'll build a community around nomad 1/2 as big as either of the aforementioned even if the software is really good.

This is a space where you can compete on operational complexity. Mesos and Kubernetes are fairly complicated to set up and run, with dependencies and intricacies that Nomad just doesn't have. From your other comment: > I do find it ironic that they talk about how nomad is for microservices and then make a dig at the several microservices that k8s is made up of. Being for microservices doesn't mean you should be a mic…

When they become bottlenecks, you scale them out. That is the beauty of the design, vs a monolith like nomad :)

It does win on setup for sure. This isn't an easy problem to solve however.

Post reply on HN