Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nomad vs. Kubernetes

nomadproject.io

Post reply on HN
Print view View original

31–40 of 369 posts

Re: Nomad vs. Kubernetes

#31

Post by AmazingTurtle »

mrweasel wrote: post #15

There seems to be plenty of reasons to run Nomad, compared to Kubernetes, but in what scenarios do Nomad lose out to Kubernetes? Is it simply a matter of Kubernetes being an open source project and Nomad being owned by HashiCorp?

Nomad vs kubernetes is like comparing apples with pears. They actually say it in their docs, that kubernetes is a complex solution whereas nomad isn't. For example when building a frontend, you don't go with react or building everything yourself, you use a framework that brings plenty of things at once that play well in an eco system. What's the matter of bringing a ton of hashicorp tools together manually? You can just use kubernetes. I'd say setting up kubernetes is just as complex as putting together a bunch of hashicorp solutions. And ultimately people (the majority) simply love all-in-one solutions

Re: Nomad vs. Kubernetes

#32

Post by mrweasel »

q3k wrote: post #27
mrweasel wrote: post #15

There seems to be plenty of reasons to run Nomad, compared to Kubernetes, but in what scenarios do Nomad lose out to Kubernetes? Is it simply a matter of Kubernetes being an open source project and Nomad being owned by HashiCorp?

Kubernetes is much more than container scheduling. Custom resources, identity and a powerful RBAC system allow you to use it as a general configuration/operational data store in your own code, from implementing operators acting upon kubernetes and the outside world to even moving most of high-level configuration glue to be natively based on Kubernetes. For example, with cert-manager running on Kubernetes you can requ…

> Kubernetes also has better support for organization-wide multi-tenant clusters than Nomad seems to have

That one's a little weird, I suppose you're right, but all the clients our Kubernetes team works with all want separate clusters or testing, staging and preproduction. They certainly don't want a multi-tenant cluster and share resources with other clients.

Re: Nomad vs. Kubernetes

#33

Post by marvinblum »

I've been running a production-grade Nomad cluster on Hetzner for the past 1 1/2 years and it's fantastic. It was amazingly easy to set up compared to Kubernetes (which I also did), the UI is awesome, updates haven't broken anything yet (as long as you follow the changelog) and it's stable. I really like the separation of concerns the HashiStack offers. You can start out just using Consul for your service meshing, and then add Nomad + Vault to get a similar experience to Kubernetes.

Yes, it doesn't cover as many features as Kubernetes, but it should be good enough for most software and you can still make the switch later. I would never go back.

You can read more on our blog if you're interested: https://pirsch.io/blog/techstack/

Re: Nomad vs. Kubernetes

#34

Post by jnsaff2 »

I liked Mesos when I worked on it, and it's been replaced by more modern tools like Nomad, but every time I have to work on k8s it's .. well it's being promoted like a cult, has a cult following and I think the whole thing is set up to suck up complexity and layering.

How do you deploy a thing to run on k8s?

One would think you deploy a manifest to it and that's it. Like yaml or json or hcl or whatever.

No. The built in thing is not good so someone wrote Helm.

So you deploy helm manifests on there?

No. Helm is not good either, you need helmsman or helmfile to further abstract things.

How do you do networking?

Layer a service mesh with custom overlay networking that abstract upon k8s clusterip.

jeesh. why?

Re: Nomad vs. Kubernetes

#35

Post by wodenokoto »

capableweb wrote: post #29
Fiahil wrote: post #25

> Kubernetes is an orchestration system for containers Yes, but No. Kubernetes is a portability platform, that happen to -also- orchestrate containers. Using kubernetes means you have complete reproducibility of the network setup, the deployment and the operation of any workload -no matter how complex-, on any kubernenetes cluster and cloud provider. Nomad, is -well- just a glorified Airflow.

Nomad, just like Unix, prefers composition of simple tools in favor of one-big-tool-for-everything that Kubernetes is going for. So for achieving those things, you'd use Terraform or something similar, and then you have a reproducible environment for the hardware/software setup outside of Nomad. > Yes, but No. Kubernetes is a portability platform, that happen to -also- orchestrate containers. The homepage of Kubernet…

> you'd use Terraform or something similar

Wouldn't you also use terraform for your kubernetes cluster?

Re: Nomad vs. Kubernetes

#37

Post by zelphirkalt »

nuerow wrote: post #19
mrweasel wrote: post #15

There seems to be plenty of reasons to run Nomad, compared to Kubernetes, but in what scenarios do Nomad lose out to Kubernetes? Is it simply a matter of Kubernetes being an open source project and Nomad being owned by HashiCorp?

> There seems to be plenty of reasons to run Nomad, compared to Kubernetes, but in what scenarios do Nomad lose out to Kubernetes? It would also be interesting if Docker's Swarm was.also featured in this comparison, as it just works right out of the box and doesn't come with any bells and whistles.

As far as I remember from trying last time a few years ago, Docker Swarm does not support rebalancing the load of containers and requires either manually issuing a command to do that or merely initially balances where containers are started. It this still true? And what about when you have state, which you would need to move between nodes, to make a container work elsewhere?

Re: Nomad vs. Kubernetes

#38

Post by q3k »

mrweasel wrote: post #32
q3k wrote: post #27

Earlier quoted context omitted.

Kubernetes is much more than container scheduling. Custom resources, identity and a powerful RBAC system allow you to use it as a general configuration/operational data store in your own code, from implementing operators acting upon kubernetes and the outside world to even moving most of high-level configuration glue to be natively based on Kubernetes. For example, with cert-manager running on Kubernetes you can requ…

> Kubernetes also has better support for organization-wide multi-tenant clusters than Nomad seems to have That one's a little weird, I suppose you're right, but all the clients our Kubernetes team works with all want separate clusters or testing, staging and preproduction. They certainly don't want a multi-tenant cluster and share resources with other clients.

> but all the clients our Kubernetes team works with all want separate clusters or testing, staging and preproduction.

And I think that's one of the biggest issues with how people use Kubernetes these days (another candidate being insisting on drive-by deploying a cluster from scratch instead of deferring to cloud providers or a dedicated platform team that can plan for long-term maintenance).

Kubernetes thrives in multi-tenant environments: you get huge resource savings and vastly simplified operations. Everyone in your organization gets access to all clusters, and they can just as easily deploy experimental best effort jobs or their development environment as they can deploy and/or inspect production jobs. Well set up quotas and priority classes mean that production jobs never run out of resources, while less important stuff (batch jobs, CI, someone's pet experiment) can continue to run on a best effort basis, just keeps getting preempted when production wants more resources.

You can even continue to have hardware separation between highly sensitive and fully untrusted jobs by using taints and tolerations, if you feel that's necessary. You still get one control plane instead of five different ones.

Re: Nomad vs. Kubernetes

#39

Post by creshal »

proxysna wrote: post #20
DeathArrow wrote: post #3

Nomad seems much simpler to use and manage if you need to do simpler things, but Kubernetes allows you to do more. We use Kubernetes instead of Nomad at work but we are also using Consul in the Kubernetes cluster.

Nomad will still get you where you need to be. Aside from vm live migration i am yet to find a good example of a workload where nomad is straight up is unable to do what you need.

What do you even still use VMs for?

Re: Nomad vs. Kubernetes

#40

Post by pdimitar »

hnlmorg wrote: post #24
tazjin wrote: post #16

Earlier quoted context omitted.

But it did not replace it. Google runs on Borg. Go lets you build things fast, but the lack of strong typing and the vast amount of language pitfalls make maintenance hard in the long run. The community also has the attitude to pretend that these pitfalls don't actually exist, which is very different from C++ where most peculiar behaviours are well-understood and controlled.

I've had just as many problems maintaining mature C++ projects as I have maintaining mature Go projects. Ultimately it all boils down to two things: - The projects being well written and well maintained from the outset - Personal preference I cannot overstate that second point. It really is about time developers stopped pushing their own personal preferences as if it's some kind of fact.

> It really is about time developers stopped pushing their own personal preferences as if it's some kind of fact.

Agreed, so let's all go to Rust or OCaml because they have very strong static typing systems. ;)

Both C++ and Go have plenty of warts, with C++ having way too many footguns and Go allowing you to use it as a dynamic language whenever you figure it's too much work to go through your problem with static types.

Post reply on HN