Live data from Hacker News

How we use HashiCorp Nomad

blog.cloudflare.com

151–160 of 166 posts

Re: How we use HashiCorp Nomad

#151
post #116

What can Nomad do that Terraform can't? I'm pretty sure you can blue-green rolling deploy services inside of Docker containers with Terraform too.

They’re completely different classes of tool - Nomad is a runtime component, Terraform is not.

but terraform can talk to the docker daemon and orchestrate containers.

what can you achieve with nomad that you can’t achieve with terraform?

Re: How we use HashiCorp Nomad

#152
post #115

Earlier quoted context omitted.

It’s also not typed SDKs over Terraform. The best Pulumi providers are native and offer a much richer experience than the wrapped Terraform providers (see the Kubernetes support for example). I for one look forward to those rolling out across the board!

I actually have one burning question if you don't mind answering: Pulumi has two different k8s modules, the standard one (pulumi-kubernetes) and "kubernetes-x" which has a much nicer API that reduces boilerplate: https://github.com/pulumi/pulumi-kubernetesx The second one doesn't seem to be used in the Pulumi tutorial content and has less stars/recognition. Are these interchangeable? If so, why not use @pulumi/kubern…

They’re not interchangeable - kubernetes-x builds higher level abstractions on top of the pulumi-kubernetes API which is faithful to the openAPI supplier by Kubernetes.

To my mind, the real power of having general purpose languages building the declarative model instead of a DSL is that libraries like k-x (or indeed AWS-x) can actually be built!

Re: How we use HashiCorp Nomad

#153
post #116

Earlier quoted context omitted.

They’re completely different classes of tool - Nomad is a runtime component, Terraform is not.

but terraform can talk to the docker daemon and orchestrate containers. what can you achieve with nomad that you can’t achieve with terraform?

- run a non-dockerised job

- have a restart policy outside of what the docker daemon offers

- automatically schedule across multiple machines based on constraints including affinity and anti-affinity, bin packing etc

- “dispatch” jobs

The Terraform Docker provider effectively provides a tiny subset of the functionality of Nomad, which provides similar functionality across many drivers, not just Docker.

Re: How we use HashiCorp Nomad

#154
post #96

Earlier quoted context omitted.

That's a fair point, I guess it depends on your use case. The risk, however, is that the powers that be at HashiCorp one day decide to abandon Nomad once they realize it will never be a profit centre for them.

The CEO addressed a similar comment on Twitter recently. From what he's saying, it seems Hashicorp is standing by Nomad for the foreseeable future. https://twitter.com/mitchellh/status/1247581788706197504

"standing by" until they're not. If their VCs start applying pressure to dump the unprofitable projects and seek more profits, their tune will change in an instant. At the end of the day these are for-profit entities, and the only thing that matters is profit.

The only reason they're pursuing this strategy is because they think they can get a piece of the kubernetes market. If that doesn't pan out, they will dump nomad like a bad habit.

For-profit companies aren't open source charities.

Re: How we use HashiCorp Nomad

#155

Earlier quoted context omitted.

It's not quite the same. Hashicorp controls whether PRs get merged. Google does not control whether PRs get merged into kubernetes. There's a long list of companies that do, including IBM, redhat, Huawei, etc. Sure, you can fork it, but now you have a separate repo that requires people to know about it.

I’m not trying to convince you. If k8s works for you, that’s great! I choose to work at places that pick Nomad instead. Best tool for the job.

Agreed, thanks!

Re: How we use HashiCorp Nomad

#156
post #132
post #121

Earlier quoted context omitted.

I can build DSLs (domain specific languages) with programming languages. I can't build DSLs with YAML. And when someone says "You will use YAML because you can't be trusted with a programming languages" then I find that very disrespectful and condescending. The solution to finite attention isn't less powerful tools. The solution is more powerful tools that augment operators and let them extend their finite attention…

A good DSL (as opposed to a raw general-purpose language) would be ideal. It's the guard rails I'm after, when implemented well. What I won't like is the "here are some classes and functions for accessing our API, go build the stuff somehow" approach.

Then we're in agreement.

Re: How we use HashiCorp Nomad

#157
post #97
post #86

I love nomad and architected a large system that used it for several years. We are finally moving away from it, however, because of its lack of native support for autoscaling. I know there are third party solutions for it, but that doesn't work for us. I suspect a large number of k8s users could use nomad instead with way less overhead.

I'm curious if you tried the new autoscaler [1] that is in Tech Preview? Is there something it's missing for your scenarios? [1] https://www.hashicorp.com/blog/hashicorp-nomad-autoscaling-t...

As you probably know, decisions like changing infrastructure architecture is a team decision and are made with the information available at the time. The nomad autoscaler just wasn't announced/released in time for us to seriously consider it. That coupled with the uncertainty around the future of Fabio made us look elsewhere. We remain staunch supporters of Hashicorp products overall. We will continue to use Terraform, Consul and a little Vault.

Re: How we use HashiCorp Nomad

#158
post #105

Do people that aren't cloudflare scale really see the need for kubernetes and/or Nomad? Of the two Nomad seems much more sane because it does one thing only and is much simpler to manage and deploy. That said, having have used it, we are mostly moving away from it. Consul + Docker/Docker-compose with systemd in "a service per vm" model has proved much easier to administrate to our scale (couple of datacenters, ~1k VM…

Interesting, I went the other direction recently, from systemd to Nomad. I was motivated by a move away from config management in favor of commands wrapping Nomad API calls. For a "devs on-call" model this was preferable to being gatekeepers of PRs against config management. To glue the whole thing together I got Consul Connect going in the Nomad jobs, so service config complexity was comparable to docker-compose. Sa…

Our devs to their own ops using our blessed ansible galaxy roles + creating their own when necessary.

It is all pretty standardized, is rare that someone needs to create a new galaxy role for example.

We expect our teams to know their way around Ansible ( which hasn’t been a problem ), we have internal docs and training

Re: How we use HashiCorp Nomad

#159

Earlier quoted context omitted.

The single binary thing is a tired trope. Kubernetes has had hyperkube for many years, which is a single binary for many years. But in really, a single binary should be at the bottom of your list of concerns.

Hyperkube is several binaries bundled in one container, no?

It's often _distributed_ as a container, but it ~~is~~ was a single monolithic go binary. However, while I was looking for supporting evidence to that claim, I discovered that it recently[1] was evicted from the github.com/k/k repo and the docker image that is now built uses a shell script named hyperkube and the Dockerfile just untars the underlying binaries into /usr/local/bin

I wasn't able to immediately track down the alleged new repo for hyperkube, though

1 = https://github.com/kubernetes/kubernetes/pull/83454

Re: How we use HashiCorp Nomad

#160

I'm surprised Hashicorp hasn't repositioned this product. Terraform was a huge breath of fresh air after Cloudformation. If you ask me, deploying apps via k8s is even better. Everyone wants a free lunch and for me, momentum + cloud vendor support + ultimately the Nomad Enterprise features that come free with K8s made the choice easy.

TF can be often suffocating still. Pulumi runs circles around it, when it comes to user experience in writing complex, modular, composable and reusable configurations.

By going with a "fork and patch" model, rather than shelling out to the existing terraform providers, they are putting themselves in a constant race to overtake TF's popularity.

Their okta provider, for example, is based on some absolutely ancient, non-standard provider (https://github.com/pulumi/pulumi-okta/blob/v2.1.2/provider/g... ) instead of using the official one which receives regular updates: https://github.com/terraform-providers/terraform-provider-ok... ; and now I guess it's on me to do some git diff to find out if there are meaningful changes between the imported provider and the upstream one?

To say nothing of the more esoteric providers that one can cheaply build and place in the `.terraform.d/plugins` directory and off you go, in contrast to trying to find the dark magick required to use some combination of https://github.com/pulumi/pulumi-terraform-bridge and https://github.com/pulumi/pulumi-tf-provider-boilerplate but ... err, ... then one has to own that new generated pulumi provider code? So like a fork but worse? Maybe they should make a bot that tracks the terraform-provider topic on GH and uses their insider knowledge to generate pulumi providers for them.

Don't get me wrong: I anxiously await the death of TF, but until there is a good story to tell my colleagues about an alternative to it, they'll continue to use it.

Post reply on HN