11–20 of 48 posts
> Nomad is designed to be a global state, optimistically concurrent scheduler. Global state means schedulers get access to the entire state of the cluster when making decisions enabling richer constraints, job priorities, resource preemption, and faster placements. Can anyone shed light on how that's possible? I was under the impression that global state in distributed systems was not possible?
Anyone able to give a compare/contrast here with other cluster management systems... Apache Mesos or Kubernetes, for example?
> Nomad is designed to be a global state, optimistically concurrent scheduler. Global state means schedulers get access to the entire state of the cluster when making decisions enabling richer constraints, job priorities, resource preemption, and faster placements. Can anyone shed light on how that's possible? I was under the impression that global state in distributed systems was not possible?
http://research.google.com/pubs/pub41684.html Basically, there's a server that holds state for the cluster. When a scheduler attempts to load a job into the cluster, it grabs state from the aforementioned server, performs its job placement calculations, and then tries to submit its answer to the master state. Since there are many such schedulers, and the amount of time it takes to place a job is non-trivial, its poss…
In most scheduling systems, two 10,000-node jobs that each would saturate a cluster on their own will time-share if submitted together, with each job acting in practice more like 10,000 single-node jobs. The result is usually each job getting a probabilistic 50% share of the cluster while they're both running, and then whichever one runs longer saturating the cluster once the other ends.
This scheduling system, meanwhile, would seem to just hand the 10,000 nodes over to job A, and then sleep job B until job A is done.
Admittedly, in the case where the jobs aren't submitted at the same time, and job A has already grabbed and saturated the cluster, the two cases collapse together: job B must wait (unless you want to schedule processes rather than containers; then you just degrade the cluster's performance.) But for batch-processing applications, you'd usually schedule everything to start at once, precisely so that the scheduler could interleave the jobs.
Also no maintenance primitives, but that's just me being in love with Mesos.
Anyone able to give a compare/contrast here with other cluster management systems... Apache Mesos or Kubernetes, for example?
Hashicorp themselves have published comparisons to Kubernetes, Mesos, et al on the Nomad site[1]. They look well written and generally not too biased. [1]: https://www.nomadproject.io/intro/vs/
I'm not sure I understand what is being said in this page.
Is the Nomad scheduler centralized? If so, it has been demonstrated that distributed scheduling (e.g. Mesos) leads to better throughput and availability, while achieving a placement close to a centralized approach.
Hashicorp is all in on Golang.
Side note to Hashicorp devs: The Products section of your homepage is virtually unreadable on Windows/Chrome: https://i.imgur.com/8st8HQk.png
