Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we use HashiCorp Nomad

blog.cloudflare.com

Post reply on HN
Print view View original

131–140 of 166 posts

Re: How we use HashiCorp Nomad

#131

Post by chucky_z »

shaklee3 wrote: post #127
chucky_z wrote: post #57

Earlier quoted context omitted.

This is what Nomad is though... It works without their other products and also natively integrates into them. Deploying Vault, Consul, and Nomad gives you a very nice experience. Also with Consul 1.8 and Nomad 0.11 you’ll get Consul Connect with Ingress gateways which solves some of those problems you mentioned.

Nomad uses consul as the kv store, so it doesn't work without their other products.

Please make an attempt to understand a technology before making comments like this. Nomad has no requirement to store kv's, nor secrets. It's an entirely separate thing from Consul and Vault. It simply integrates with Consul, and Vault. It even runs its own raft store, so each product's backend is totally separate (Vault dropped it's reliance on other backends as of 1.4 and can run it's own raft store now).

Nomad template stanzas are simply consul-template (https://github.com/hashicorp/consul-template#plugins), you could use `{{ with secret "" }}` and never touch Consul. You also have every function beyond service and the key* set of functions. You could build pretty static, or dynamic configurations using these blocks without ever touching Consul. On top of that, both Terraform templates and Levant work well for templating job specs out themselves, which contain template stanzas in them.

An example of something helpful would be if you wanted to drop a very small binary that changes with each deploy, you could use https://github.com/hashicorp/consul-template#base64decode and just change the contents each time you deploy the job.

If you wanted to use redis keys in your consul-template, simply drop a binary on each server as a plugin to consul-template, then for example: `{{ "my-key" | plugin "redis" "get" }}`.

Why not try running it yourself?

`nomad agent -dev` will get you a server running, then `nomad job init` will give you a full spec which you can run with `nomad job run example.job`.

Re: How we use HashiCorp Nomad

#132

Post by nine_k »

memexy wrote: post #121
nine_k wrote: post #120

Earlier quoted context omitted.

A full programming language gives too much flexibility. In particular, this includes the flexibility allowing to shoot oneself in the foot. A person who constantly works with computers is painfully aware of the people's ability to screw things up by an honest mistake when programming them. The more complex the thing, the easier it is to make a mistake, because human attention is finite. Whenever you can limit the lan…

I can build DSLs (domain specific languages) with programming languages. I can't build DSLs with YAML. And when someone says "You will use YAML because you can't be trusted with a programming languages" then I find that very disrespectful and condescending. The solution to finite attention isn't less powerful tools. The solution is more powerful tools that augment operators and let them extend their finite attention…

A good DSL (as opposed to a raw general-purpose language) would be ideal. It's the guard rails I'm after, when implemented well.

What I won't like is the "here are some classes and functions for accessing our API, go build the stuff somehow" approach.

Re: How we use HashiCorp Nomad

#133

Post by takeda »

otterley wrote: post #65
4636760295 wrote: post #48

Earlier quoted context omitted.

That's a fair point, I guess it depends on your use case. The risk, however, is that the powers that be at HashiCorp one day decide to abandon Nomad once they realize it will never be a profit centre for them.

Nomad is open source (or, at least, a significant subset of it is). Anyone is able to continue to improve it, even if Hashicorp is no longer paying people to work on it.

That's not enough, Basho (creators of Riak database) also made it open source. In fact after they went out of business Bet365 even purchased their proprietary code and made it open source, but the database is still considered dead.

Re: How we use HashiCorp Nomad

#134

Post by meitham »

rayanimesh wrote: post #30
ognyankulev wrote: post #11

Earlier quoted context omitted.

Helm charts are declarative way of deploying app(s) and their accompanying resources.

> Helm charts are declarative way of deploying app(s) and their accompanying resources. How do you make helm chart deployment declarative? `helm install` is not declarative (in my understanding `kubectl apply` is declarative and `kubectl create` is not. Let me know if my understanding of declarative is wrong). Thanks.

I think you’re confusing “declarative” with “idempotent”! The helm equivalent to “kubectl apply” is “helm upgrade —-install”.

Re: How we use HashiCorp Nomad

#135

Post by jordanbeiber »

solatic wrote: post #118

So basically, when CloudFlare was making the decision to adopt Nomad, they had already adopted Consul and had already built in-house a custom scheduler (unimog) for their customer traffic. It's rather disingenuous to compare to Kubernetes-based installations by pointing out that Nomad is a single Go binary that's easy to install, because they're also running Consul and unimog. For what it's worth, kubelet (which sche…

Having worked extensively with both the k8s and hashicorp stack let me tell you that there is something to be said regarding the simplicity of the former.

First - companies will want service discovery for all managed resources. Consul is just outstanding, so there’s that.

On-prem, nomad is simple to manage and it gives you a form of freedom that k8s locks up in a black box.

After all, it’s a process scheduler, go ahead and schedule what you want. A container, raw exec or batch job. Add parameters to the batch job and dispatch them at will.

If you start from scratch or can move absolutely everything to k8s? Well sure, but not a lot of places you’d call “enterprise” isn’t in that position.

Re: How we use HashiCorp Nomad

#136

Post by jordanbeiber »

shaklee3 wrote: post #122
solatic wrote: post #118

So basically, when CloudFlare was making the decision to adopt Nomad, they had already adopted Consul and had already built in-house a custom scheduler (unimog) for their customer traffic. It's rather disingenuous to compare to Kubernetes-based installations by pointing out that Nomad is a single Go binary that's easy to install, because they're also running Consul and unimog. For what it's worth, kubelet (which sche…

The single binary thing is a tired trope. Kubernetes has had hyperkube for many years, which is a single binary for many years. But in really, a single binary should be at the bottom of your list of concerns.

Hyperkube is several binaries bundled in one container, no?

Re: How we use HashiCorp Nomad

#137

Post by dilyevsky »

q3k wrote: post #62
threentaway wrote: post #54

Earlier quoted context omitted.

If you like those, I'd take a look at Grafana's Tanka [0]. It also uses jsonnet but has some additional features such as showing a diff of your changes before you apply, easy vendored libraries using jsonnet-bundler, and the concept of "environments" which prevents you from accidentally applying changes to the wrong namespace/cluster. [0] https://github.com/grafana/tanka

I looked at it, I don't like it for the same reason as I dislike many other tools in this space: it imposes its own directory structure, abstraction (environments) and workflow. I'm a fan of the kubecfg-style approach, where it lets you use whatever sort of structure makes sense for you and your project. It's a 'framework' vs 'library' thing, but in the devops context.

I worked on this at previous gig - https://github.com/cruise-automation/isopod

Same use-case but uses Starlark dsl instead of jsonnet

Re: How we use HashiCorp Nomad

#138

Post by rayanimesh »

meitham wrote: post #134
rayanimesh wrote: post #30

Earlier quoted context omitted.

> Helm charts are declarative way of deploying app(s) and their accompanying resources. How do you make helm chart deployment declarative? `helm install` is not declarative (in my understanding `kubectl apply` is declarative and `kubectl create` is not. Let me know if my understanding of declarative is wrong). Thanks.

I think you’re confusing “declarative” with “idempotent”! The helm equivalent to “kubectl apply” is “helm upgrade —-install”.

Yes. Thank you.

Re: How we use HashiCorp Nomad

#139

Post by alledm »

ianlevesque wrote: post #9

After trying out most of the kubernetes ecosystem in the pursuit of a declarative language to describe and provision services, Nomad was a breath of fresh air. It is so much easier to administer and the nomad job specs were exactly what I was looking for. I also noticed a lot of k8s apps encourage the use of Helm or even shell scripts to set up key pieces, which defeats the purpose if you are trying to be declarative…

https://kapitan.dev/ is the one-stop shop that covers for true declarative configuration with either jsonnet, python (kadet) and jinja, amazing secret management with support of gkms, awskms, gpg, vault.

It is simpler than other tools, because you can get started without even touching jsonnet or python or anything else, when using our generators.

It does more than all the other tools combined, as it replaces helm+helmfile+gitcrypt or kustomize.

It's universal, so you can use it on non-kubernetes situations where other tools leave you high and dry.

With the new generator library, you can have 1 template and use it to configure many services. Check our examples at https://github.com/kapicorp/kapitan-reference

We have just released: * https://github.com/kapicorp/kapitan-reference a repo with examples for quick-start. It includes our generator as explained in https://medium.com/kapitan-blog/keep-your-ship-together-with.... * https://github.com/kapicorp/tesoro a secret "webhook controller" to seamlessly handle Kapitan secrets in your cluster. Better than sealed-secrets because there is no need to convert secrets and it supports KMS like google and aws together.

Get started with our blog: https://medium.com/kapitan-blog or join our kubernetes slack on #kapitan

Post reply on HN