After trying out most of the kubernetes ecosystem in the pursuit of a declarative language to describe and provision services, Nomad was a breath of fresh air. It is so much easier to administer and the nomad job specs were exactly what I was looking for. I also noticed a lot of k8s apps encourage the use of Helm or even shell scripts to set up key pieces, which defeats the purpose if you are trying to be declarative…
Helm charts are declarative way of deploying app(s) and their accompanying resources.
After trying out most of the kubernetes ecosystem in the pursuit of a declarative language to describe and provision services, Nomad was a breath of fresh air. It is so much easier to administer and the nomad job specs were exactly what I was looking for. I also noticed a lot of k8s apps encourage the use of Helm or even shell scripts to set up key pieces, which defeats the purpose if you are trying to be declarative…
Helm charts are declarative way of deploying app(s) and their accompanying resources.
Helm, however, is objectively terrible with its yaml-based templating language and zero practical modularity.
Off topic but... I've always wondered how Cloudflare can not charge for the bandwidth. Even the free plan is super generous (CDN, SSL, etc). How are they making a profit when AFAIK all other CDNs charge you for the bandwidth (and I assume they have to pay for to their providers)?
Which features is Nomad missing? Feature count comparisons are meaningless unless the features are tied to actual important use cases. Lots of software is encrusted with rarely used features that just add complexity.
The free version of Nomad is missing, IIUC: - no quotas for teams/projects/organizations - no preemption (ie higher priority job preempts lower priority job) - no namespacing So generally it's somewhat useless in organizations where there are multiple different teams that should be able to coexist on a cluster without stepping on eachothers' toes, or even where you want a CI system to access the cluster in a safe man…
Preemption is coming in the next release in OSS :)
Which features is Nomad missing? Feature count comparisons are meaningless unless the features are tied to actual important use cases. Lots of software is encrusted with rarely used features that just add complexity.
The free version of Nomad is missing, IIUC: - no quotas for teams/projects/organizations - no preemption (ie higher priority job preempts lower priority job) - no namespacing So generally it's somewhat useless in organizations where there are multiple different teams that should be able to coexist on a cluster without stepping on eachothers' toes, or even where you want a CI system to access the cluster in a safe man…
We run everything in Nomad for all our teams, but we don’t use any of the features you mention, and it’s not causing issues. We wrote some metrics for how much memory all the allocations are claiming, and we throw another EC2 instance in the cluster when we’re running out. Works pretty well. Preemption seems like a nice feature, but I imagine getting teams to coordinate their preemption values would be a political nightmare.
Which features is Nomad missing? Feature count comparisons are meaningless unless the features are tied to actual important use cases. Lots of software is encrusted with rarely used features that just add complexity.
The free version of Nomad is missing, IIUC: - no quotas for teams/projects/organizations - no preemption (ie higher priority job preempts lower priority job) - no namespacing So generally it's somewhat useless in organizations where there are multiple different teams that should be able to coexist on a cluster without stepping on eachothers' toes, or even where you want a CI system to access the cluster in a safe man…
We fully intend to migrate more features to OSS in the future -- especially as we build out more enterprise features. As you can imagine building a sustainable business is quite the balancing act, and there's constant internal discussion.
The free version of Nomad is missing, IIUC: - no quotas for teams/projects/organizations - no preemption (ie higher priority job preempts lower priority job) - no namespacing So generally it's somewhat useless in organizations where there are multiple different teams that should be able to coexist on a cluster without stepping on eachothers' toes, or even where you want a CI system to access the cluster in a safe man…
We run everything in Nomad for all our teams, but we don’t use any of the features you mention, and it’s not causing issues. We wrote some metrics for how much memory all the allocations are claiming, and we throw another EC2 instance in the cluster when we’re running out. Works pretty well. Preemption seems like a nice feature, but I imagine getting teams to coordinate their preemption values would be a political ni…
For me the ability to squeeze in some jobs in between the cracks at best effort priority is crucial (ie. any sort of high latency batch processing / experiments). I wouldn't want these extremely low priority jobs to compete with, I don't know, an actual customer facing service.
Also, we run on bare metal, so there really isn't a way to request extra capacity within seconds.
Which features is Nomad missing? Feature count comparisons are meaningless unless the features are tied to actual important use cases. Lots of software is encrusted with rarely used features that just add complexity.
Does it have "easy" side-cars/init/post? Last time I checked those were missing for example.
It does! Here is the relevant documentation and examples:
Cloudflare, make sure you upgrade to 0.11.3, the new scheduling behavior is awesome for large clusters. Also a massive warning to anyone wanting to use hard cpu limits and cgroups (they do work in nomad it’s just not trivial), they don’t work like anyone expects and need to be heavily tested.
are you serious? you've got a major company touting a technology and a key component of it is broken?