Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nomad v1.0 release – workload orchestration

github.com

Post reply on HN
Print view View original

11–20 of 27 posts

Re: Nomad v1.0 release – workload orchestration

#11

Post by gavinray »

schmichael wrote: post #7
gavinray wrote: post #4

Earlier quoted context omitted.

I'm a big fan conceptually of Nomad, and I love the streams that the Nomad team does to talk about development and answer questions. My only comments would be: I wish there was more content available (maybe on HashiCorp Learn Nomad) for working with single-instance Nomad clusters. And a demo of realworld dev-to-prod workflow. Something like "Okay here's a local Docker Compose setup with Postgres, a backend API, and a…

Thanks that's great feedback and you're not alone in that desire. We do "small" users a pretty big disservice by effectively dismissing single-server deployments and jumping straight to "real" production deployments (3+ servers distinct from the nodes that run workloads). We have people who use Nomad locally as a systemd alternative. We have people who use single-scheduler-multiple-nodes clusters at home or at work,…

Yeah, I mean also from a financial/business standpoint I totally get it.

The incentive is to target content efforts towards enterprise and large-scale orgs. Because who cares if the indie dev or small startup is running your stack, let's be real.

Re: Nomad v1.0 release – workload orchestration

#12

Post by jrnkntl »

schmichael wrote: post #2

There's a blog post as well. I'm unsure what makes for the most relevant link: https://www.hashicorp.com/blog/announcing-general-availabili... I'm the Nomad Team Engineering Lead and would be happy to answer any questions people might have.

Ah, I've submitted this as soon as I saw the notification of the new release on Github, maybe the blog post was indeed more relevant.

Congratulations on this milestone release, we're using Nomad since March this year on a single 'bare-metal' server and it serves our needs perfectly. We set it up with a simple gorilla/mux API in front and use the Nomad API to submit jobs from all our other applications and it works flawlessly.

With regards to 1.0's features:

HCL2 is a welcome addition for us since we had a lot of repetition in our job files using artifacts for tasks.

Also the addition of the PostStop lifecycle couldn't come at a better time, we were discussing workarounds for this recently.

One area of potential improvement would be the behaviour of file/directory permissions through different task drivers, I know this is totally dependent of the different drivers we can use with Nomad but more than once we bumped into this while setting up our jobs (and others too [1][2])

[1] https://github.com/hashicorp/nomad/issues/2625 [2] https://github.com/hashicorp/nomad/issues/8892

Thanks for all the work your team did. I am a big fan of the Hashicorp ecosystem.

Re: Nomad v1.0 release – workload orchestration

#13

Post by hedwall »

ofrzeta wrote: post #9
mike1o1 wrote: post #3

I really like the idea of Nomad as a simpler alternative to Kubernetes. The documentation is great and provides a lot of examples, but I wish I could find a starter guide on how to deploy a complete application with it and Consul. Something like, here’s how to deploy a Rails app with Sidekiq workers and database migrations. This along with Waypoint seems like a great solution for smaller side projects.

I like it, too, but with the integration of CNI it's no longer so much simpler than Kubernetes. Where Kubernetes has etcd Nomad requires Consul which arguably might be simpler. To compete with Kubernetes Nomad will most likely acquire more features (e.g. storage) until it's as complex as Kubernetes because it's just what some businesses seem to want. On the other hand Kubernetes gets simpler with projects like k3s or…

There's no requirement on Consul for deploying Nomad, and storage support is in Beta right now.

Re: Nomad v1.0 release – workload orchestration

#14

Post by schmichael »

ofrzeta wrote: post #9
mike1o1 wrote: post #3

I really like the idea of Nomad as a simpler alternative to Kubernetes. The documentation is great and provides a lot of examples, but I wish I could find a starter guide on how to deploy a complete application with it and Consul. Something like, here’s how to deploy a Rails app with Sidekiq workers and database migrations. This along with Waypoint seems like a great solution for smaller side projects.

I like it, too, but with the integration of CNI it's no longer so much simpler than Kubernetes. Where Kubernetes has etcd Nomad requires Consul which arguably might be simpler. To compete with Kubernetes Nomad will most likely acquire more features (e.g. storage) until it's as complex as Kubernetes because it's just what some businesses seem to want. On the other hand Kubernetes gets simpler with projects like k3s or…

Nomad has supported CSI for storage since 0.11. It is definitely a challenge since despite being an orchestrator agnostic standard, storage vendors often assume k8s and only support it.

That being said Nomad's CSI support doesn't impact clusters that don't optin to use it. Jobs that use host volumes or ephemeral volumes still work. Only using Nomad for stateless workloads still works. We try very hard to introduce new features in a way that only impacts people who use them.

While the principle is the same for CNI, our migration to group networks ("groups" in Nomad are like "pods" in k8s) and away from task networks has been more painful than we had hoped. Existing jobs should still work with task networks and we're rapidly trying to fix differences in the two approaches.

Nomad's Consul dependency does introduce complexity. The migration to group networks actually included a change that made service addressing available to servers in such a way that Nomad could offer native service discovery. It's still being discussed whether we want to pursue that since offering multiple solutions has obvious downsides as well.

Re: Nomad v1.0 release – workload orchestration

#15

Post by schmichael »

jrnkntl wrote: post #12
schmichael wrote: post #2

There's a blog post as well. I'm unsure what makes for the most relevant link: https://www.hashicorp.com/blog/announcing-general-availabili... I'm the Nomad Team Engineering Lead and would be happy to answer any questions people might have.

Ah, I've submitted this as soon as I saw the notification of the new release on Github, maybe the blog post was indeed more relevant. Congratulations on this milestone release, we're using Nomad since March this year on a single 'bare-metal' server and it serves our needs perfectly. We set it up with a simple gorilla/mux API in front and use the Nomad API to submit jobs from all our other applications and it works fl…

> One area of potential improvement would be the behaviour of file/directory permissions through different task drivers, I know this is totally dependent of the different drivers we can use with Nomad but more than once we bumped into this while setting up our jobs (and others too [1][2])

Thanks for mentioning these. Everyone interested should definitely +1 them as we do use reaction emoji during prioritization.

The task driver dependent nature does make this tricky, but since the Nomad agent is usually run as root and controls the allocation/task directory tree we should have some options here.

Re: Nomad v1.0 release – workload orchestration

#16

Post by ofrzeta »

schmichael wrote: post #14
ofrzeta wrote: post #9

Earlier quoted context omitted.

I like it, too, but with the integration of CNI it's no longer so much simpler than Kubernetes. Where Kubernetes has etcd Nomad requires Consul which arguably might be simpler. To compete with Kubernetes Nomad will most likely acquire more features (e.g. storage) until it's as complex as Kubernetes because it's just what some businesses seem to want. On the other hand Kubernetes gets simpler with projects like k3s or…

Nomad has supported CSI for storage since 0.11. It is definitely a challenge since despite being an orchestrator agnostic standard, storage vendors often assume k8s and only support it. That being said Nomad's CSI support doesn't impact clusters that don't optin to use it. Jobs that use host volumes or ephemeral volumes still work. Only using Nomad for stateless workloads still works. We try very hard to introduce ne…

Thanks for your comprehensive answer. As I said I like Nomad and in fact I'd probably prefer to use Nomad if Kubernetes hadn't become kind of an industry standard. When people are free to choose they should by all means use Nomad :)

That said Kubernetes is also simple with stateless workloads or host local storage. It becomes complex when you are using some kind of cluster managed storage and I guess there will be demand for it on Nomad (when you are lucky) and there will be no way around that complexity.

Eventually there will also be things like Nomad Operators and such to handle the increasing complexity :)

Anyway keep up the good work!

Re: Nomad v1.0 release – workload orchestration

#17

Post by ngrilly »

schmichael wrote: post #7
gavinray wrote: post #4

Earlier quoted context omitted.

I'm a big fan conceptually of Nomad, and I love the streams that the Nomad team does to talk about development and answer questions. My only comments would be: I wish there was more content available (maybe on HashiCorp Learn Nomad) for working with single-instance Nomad clusters. And a demo of realworld dev-to-prod workflow. Something like "Okay here's a local Docker Compose setup with Postgres, a backend API, and a…

Thanks that's great feedback and you're not alone in that desire. We do "small" users a pretty big disservice by effectively dismissing single-server deployments and jumping straight to "real" production deployments (3+ servers distinct from the nodes that run workloads). We have people who use Nomad locally as a systemd alternative. We have people who use single-scheduler-multiple-nodes clusters at home or at work,…

I’d be happy to know more about the single node use case. I considered Nomad in the past for this, but it seemed optimized for a cluster, and I ended up using Docker Swarm instead.

Re: Nomad v1.0 release – workload orchestration

#18

Post by ngrilly »

schmichael wrote: post #7
gavinray wrote: post #4

Earlier quoted context omitted.

I'm a big fan conceptually of Nomad, and I love the streams that the Nomad team does to talk about development and answer questions. My only comments would be: I wish there was more content available (maybe on HashiCorp Learn Nomad) for working with single-instance Nomad clusters. And a demo of realworld dev-to-prod workflow. Something like "Okay here's a local Docker Compose setup with Postgres, a backend API, and a…

Thanks that's great feedback and you're not alone in that desire. We do "small" users a pretty big disservice by effectively dismissing single-server deployments and jumping straight to "real" production deployments (3+ servers distinct from the nodes that run workloads). We have people who use Nomad locally as a systemd alternative. We have people who use single-scheduler-multiple-nodes clusters at home or at work,…

For single node or small cluster deployments, one issue with k8s is the significant CPU and RAM usage even on a fully idle system with no payload. How is Nomad in that regard? Are the controllers also implemented as a control loop that is constantly polling and diffing reported and desired state?

Re: Nomad v1.0 release – workload orchestration

#20

Post by khimaros »

ofrzeta wrote: post #9
mike1o1 wrote: post #3

I really like the idea of Nomad as a simpler alternative to Kubernetes. The documentation is great and provides a lot of examples, but I wish I could find a starter guide on how to deploy a complete application with it and Consul. Something like, here’s how to deploy a Rails app with Sidekiq workers and database migrations. This along with Waypoint seems like a great solution for smaller side projects.

I like it, too, but with the integration of CNI it's no longer so much simpler than Kubernetes. Where Kubernetes has etcd Nomad requires Consul which arguably might be simpler. To compete with Kubernetes Nomad will most likely acquire more features (e.g. storage) until it's as complex as Kubernetes because it's just what some businesses seem to want. On the other hand Kubernetes gets simpler with projects like k3s or…

Podman can also deploy from Kubernetes config https://github.com/containers/podman/blob/master/docs/source...
Post reply on HN