Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we use HashiCorp Nomad

blog.cloudflare.com

Post reply on HN
Print view View original

101–110 of 166 posts

Re: How we use HashiCorp Nomad

#101

Post by syllogism »

jambay wrote: post #95
syllogism wrote: post #39

Earlier quoted context omitted.

A few long-standing feature requests I've noticed a lot: * No autoscaling * Can't reserve entire CPU core: https://github.com/hashicorp/nomad/issues/289 * No way to run jobs sequentially: https://github.com/hashicorp/nomad/issues/419

Autoscaling is in Tech Preview as of March: https://www.hashicorp.com/blog/hashicorp-nomad-autoscaling-t... Disclaimer: I work at HashiCorp in Product Management.

Nice! It's definitely been one of the bigger holes.

I hope you'll also consider sequential tasks in a single jobfile. Without it it's kind of awkward because you have to be passing shell scripts around just to run a setup step before your actual workload.

Re: How we use HashiCorp Nomad

#102

Post by gavinray »

jen20 wrote: post #99
tetha wrote: post #81

Earlier quoted context omitted.

Except now you have to deal with javascript. It's hard enough to turn some operators towards an IaC approach, but throwing Javascript at them isn't going to help.

Pulumi does not require JavaScript at all - it is one of a range of options, including JavaScript, TypeScript, Python, Go, C# and F# (currently). Disclaimer: I worked on Terraform at HashiCorp and am a contributor to Pulumi also.

Pulumi is so good (even if it is just typed-SDK's over Terraform). Only sane approach to IaC IMO, using an actual programming language with types and IDE integration.

Really dig using with it Cloud Run.

Re: How we use HashiCorp Nomad

#103

Post by ridv »

luhn wrote: post #96
4636760295 wrote: post #48

Earlier quoted context omitted.

That's a fair point, I guess it depends on your use case. The risk, however, is that the powers that be at HashiCorp one day decide to abandon Nomad once they realize it will never be a profit centre for them.

The CEO addressed a similar comment on Twitter recently. From what he's saying, it seems Hashicorp is standing by Nomad for the foreseeable future. https://twitter.com/mitchellh/status/1247581788706197504

Nit: Mitchell is their CTO. Dave McJannet is their CEO. (source: https://www.hashicorp.com/about/ )

Re: How we use HashiCorp Nomad

#104

Post by nickethier »

syllogism wrote: post #101
jambay wrote: post #95

Earlier quoted context omitted.

Autoscaling is in Tech Preview as of March: https://www.hashicorp.com/blog/hashicorp-nomad-autoscaling-t... Disclaimer: I work at HashiCorp in Product Management.

Nice! It's definitely been one of the bigger holes. I hope you'll also consider sequential tasks in a single jobfile. Without it it's kind of awkward because you have to be passing shell scripts around just to run a setup step before your actual workload.

We’ve also started to implement this with a new task dependencies feature in the latest release. Check out: https://www.hashicorp.com/resources/preview-of-nomad-0-11-ta...

https://learn.hashicorp.com/nomad/task-deps/interjob

There’s more work going on there to further improve the feature in upcoming releases.

Disclaimer: I’m an engineer on the Nomad team.

Re: How we use HashiCorp Nomad

#105

Post by aprdm »

Do people that aren't cloudflare scale really see the need for kubernetes and/or Nomad?

Of the two Nomad seems much more sane because it does one thing only and is much simpler to manage and deploy.

That said, having have used it, we are mostly moving away from it. Consul + Docker/Docker-compose with systemd in "a service per vm" model has proved much easier to administrate to our scale (couple of datacenters, ~1k VM mark). It is actually so simple that is boring. Instead of fiddling with infrastructure the developers spend time solving business problems...

Our stack is really boring, developers can find their way around ansible and share galaxy roles to configure / provision infrastructure... Other cloud native projects like Prometheus and Fluentd give us all the visibility we need in a very straight forward and boring way.

p.s: Great article!

Re: How we use HashiCorp Nomad

#106

Post by macintux »

memexy wrote: post #98
tetha wrote: post #81

Earlier quoted context omitted.

Except now you have to deal with javascript. It's hard enough to turn some operators towards an IaC approach, but throwing Javascript at them isn't going to help.

What language would be good for operators if not JavaScript?

Speaking as a 25-year sys/netadmin who writes a fair bit of code too: nearly anything.

I'd rather learn Go than deal with JavaScript. Python would be fine. Elixir would be great (I'd much prefer Erlang but there are only so many miracles I'm allotted in this lifetime). Perl, please.

Re: How we use HashiCorp Nomad

#107

Post by shanemhansen »

aprdm wrote: post #105

Do people that aren't cloudflare scale really see the need for kubernetes and/or Nomad? Of the two Nomad seems much more sane because it does one thing only and is much simpler to manage and deploy. That said, having have used it, we are mostly moving away from it. Consul + Docker/Docker-compose with systemd in "a service per vm" model has proved much easier to administrate to our scale (couple of datacenters, ~1k VM…

Imo kubernetes isn't about "scale". Folks like cloudflare often run a very limited number of services (proxy plus control plane), which can make features like quotas/affinities/limits less useful because 100% of the machine installed in the pop is dedicated to one process.

I see kubernetes as more useful when organizationally you have more heterogeneous services and separate dev/sre groups. This allows them to divide up responsibilities more easily.

Re: How we use HashiCorp Nomad

#109

Post by halbritt »

rochacon wrote: post #56
ianlevesque wrote: post #23

Earlier quoted context omitted.

Maybe I was doing it wrong but every guide was verb based - “helm install X”. My declarative ideal ended up being a text file full of helm install commands and that wasn’t what I wanted.

Quick-start guides takes the easiest path to get something running, which is `helm install` in the Helm world. If you want to have complete control of what you're pushing to the API, use Helm as an starting point instead, run `helm template` and save the YAML output to some file, publish it using `kubectl` or some other rollout tool. I recommend using `kapp` [1] for rollouts. [1] https://get-kapp.io/

This is precisely how Spinnaker behaves with Helm. It renders the template into manifests and then deploys them.

Kinda loses the benefit of Helm hooks, but if one is using Spinnaker, there are other ways to do the same thing hooks can do.

Re: How we use HashiCorp Nomad

#110

Post by halbritt »

shanemhansen wrote: post #107
aprdm wrote: post #105

Do people that aren't cloudflare scale really see the need for kubernetes and/or Nomad? Of the two Nomad seems much more sane because it does one thing only and is much simpler to manage and deploy. That said, having have used it, we are mostly moving away from it. Consul + Docker/Docker-compose with systemd in "a service per vm" model has proved much easier to administrate to our scale (couple of datacenters, ~1k VM…

Imo kubernetes isn't about "scale". Folks like cloudflare often run a very limited number of services (proxy plus control plane), which can make features like quotas/affinities/limits less useful because 100% of the machine installed in the pop is dedicated to one process. I see kubernetes as more useful when organizationally you have more heterogeneous services and separate dev/sre groups. This allows them to divide…

Increases velocity as well.

I have a set of developers that can focus on their code and not have to worry about the infrastructure, TLS certs, DNS, storage or whatever. That all gets abstracted away.

For SRE, that gives one a common set of orchestration tooling one can develop against.

Post reply on HN