Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nomad vs. Kubernetes

nomadproject.io

Post reply on HN
Print view View original

61–70 of 369 posts

Re: Nomad vs. Kubernetes

#61

Post by glynnforrest »

> Flexible Workload Support

This is Nomad's most underrated feature, IMHO. You don't have to use containers for everything if you don't want to. For example, if you're a golang shop you can run everything as native binaries and cut out docker completely.

Nomad has much simpler networking, i.e. no web of iptables rules to figure out. You can add Consul connect as a service mesh if you need it, but if you don't, you can keep things very simple. Simple = easy to understand, run, and debug.

The main downside for me is a lack of plug and play pieces, e.g. a helm chart for Grafana or Prometheus. You'll have to write your own job for those, though it's very easy to learn. I'd love to see a series of Nomad recipes that people could use.

I think it's the ideal choice for on-prem, bare-metal, or 'weird' deployments where you need a bit more control. You can build the exact stack you need with the different HashiCorp projects with minimal overhead.

I can't recommend it enough! I help people move to Nomad, my email is in my profile if you want to chat :)

Re: Nomad vs. Kubernetes

#62

Post by nirn »

streetcat1 wrote: post #53
sjtindell wrote: post #13

Earlier quoted context omitted.

In my experience you end up needing additional tools for everything in Kubernetes such as secrets, auth, etc. that’s the pluggable API selling point.

Might. But most of them are open source with healthy competition. With Nomad, the ecosystem is much smaller(basically one company), and can always be closed under a paywall (e.g. terraform enterprise)

Do you mean Nomad Enterprise? Which actually exist https://www.nomadproject.io/docs/enterprise :)

Re: Nomad vs. Kubernetes

#63

Post by acdha »

q3k wrote: post #38
mrweasel wrote: post #32

Earlier quoted context omitted.

> Kubernetes also has better support for organization-wide multi-tenant clusters than Nomad seems to have That one's a little weird, I suppose you're right, but all the clients our Kubernetes team works with all want separate clusters or testing, staging and preproduction. They certainly don't want a multi-tenant cluster and share resources with other clients.

> but all the clients our Kubernetes team works with all want separate clusters or testing, staging and preproduction. And I think that's one of the biggest issues with how people use Kubernetes these days (another candidate being insisting on drive-by deploying a cluster from scratch instead of deferring to cloud providers or a dedicated platform team that can plan for long-term maintenance). Kubernetes thrives in m…

> You can even continue to have hardware separation between highly sensitive and fully untrusted jobs by using taints and tolerations, if you feel that's necessary. You still get one control plane instead of five different ones.

How much have you had that setup audited? It seems like a lot people aren’t comfortable saying that the internal boundaries are strong enough, which leads to the proliferation of separate clusters.

Re: Nomad vs. Kubernetes

#64

Post by indigochill »

sheeshkebab wrote: post #57
heinrichhartman wrote: post #41

Earlier quoted context omitted.

> How do you deploy a thing to run on k8s? kubectrl apply -f deploy.yaml should work, no? What forces you to use the sugar-coating?

Maybe bc no one uses that in reality and use helm instead?

I'll take that "no one" badge. I've never used Helm, always used kubectl.

Well, until now, when I'm using Terraform (particularly helpful relative to kubectl when I need to provision cloud resources directly), but I've still never used Helm.

Re: Nomad vs. Kubernetes

#65

Post by musingsole »

jjeaff wrote: post #42
jnsaff2 wrote: post #34

I liked Mesos when I worked on it, and it's been replaced by more modern tools like Nomad, but every time I have to work on k8s it's .. well it's being promoted like a cult, has a cult following and I think the whole thing is set up to suck up complexity and layering. How do you deploy a thing to run on k8s? One would think you deploy a manifest to it and that's it. Like yaml or json or hcl or whatever. No. The built…

You don't need any of that stuff. When I started with k8s, all I used were Yaml manifest files and kubectl apply. As I started using it more, i eventually moved up to using helm. I've been running production k8s for a few years now and haven't used helmsman or anything but helm yet.

> When I started with k8s

But that's the problem right there. Shop after shop has invested in kubernetes with a team that was just learning it. And then they layered tool after tool to help massage pain points -- of which kubernetes has plenty. That leaves us at today where every shop effectively has a custom stack of kubernetes infrastructure cobbled together of various tools and philosophies.

It's the same problem as JavaScript. There's nothing terrible about the technology -- on the contrary it's amazing stuff! But the context of how it gets used leads to a type of gridlock of opinionated tooling published just last week.

Re: Nomad vs. Kubernetes

#66

Post by GlennS »

EdwardDiego wrote: post #54
_hzrk wrote: post #5

Earlier quoted context omitted.

> Being written in a language that isn't well-suited to large projects and refactoring efforts doesn't help either. I know that Borg was written in Java and Kubernetes in Go. Though the latter had a reputation in the beginning as a systems programming language, its purpose was actually to build large-scale cloud infrastructure projects with it and it proved formidably well suited for the task. It compiles fast, anyon…

I just wish klog would go die in a fire, I'm so confused at why logging is so terrible in Go.

Can you expand on this?

My experience with Go has just been making small changes to small programs. So, I don't know what the normal experience is.

My experience with logging varies from:

print (works fine I guess)

import logging (this is pretty good - means I don't have to parse my logs before deciding where to send them)

import slf4j (6 logging frameworks and a logging framework disintermediation framework)

Re: Nomad vs. Kubernetes

#67

Post by _wldu »

I have never deployed or built a K8s cluster, but recently I moved about a dozen AWS ECS Fargate workloads into a K8s cluster that a colleague of mine has setup. I was surprised. I really like it (from the perspective of a user/developer). I deploy my apps by updating a simple yaml file in a git repo. All my other practices (vim, go code with standard Makefiles, docker container image registry) are unchanged.

I also think K8s is a reasonable cloud independent abstraction when you need to move your workloads to another provider. It prevents cloud lock-in. And I suppose Nomad would do that too.

So far my K8s experience has been very good. However, if I ever have to do more with it (on the administrative side) I may have a less positive experience. Not sure.

Re: Nomad vs. Kubernetes

#69

Post by acdha »

Fiahil wrote: post #48
capableweb wrote: post #29

Earlier quoted context omitted.

Nomad, just like Unix, prefers composition of simple tools in favor of one-big-tool-for-everything that Kubernetes is going for. So for achieving those things, you'd use Terraform or something similar, and then you have a reproducible environment for the hardware/software setup outside of Nomad. > Yes, but No. Kubernetes is a portability platform, that happen to -also- orchestrate containers. The homepage of Kubernet…

> The homepage of Kubernetes seems to disagree with you. People have trouble understanding what k8s is, and what to use it for. That's fine, it'll take a while, but they will eventually understand what "Production-Grade Container Orchestration" really means when they start working with it.

Can you explain this? Having read it I have no idea what you think the differences are — it just sounds smug.

Re: Nomad vs. Kubernetes

#70

Post by musingsole »

EdwardDiego wrote: post #58
jnsaff2 wrote: post #34

I liked Mesos when I worked on it, and it's been replaced by more modern tools like Nomad, but every time I have to work on k8s it's .. well it's being promoted like a cult, has a cult following and I think the whole thing is set up to suck up complexity and layering. How do you deploy a thing to run on k8s? One would think you deploy a manifest to it and that's it. Like yaml or json or hcl or whatever. No. The built…

Nothing about K8s forces you to use Helm or istio etc.

No, the developers about k8s force you to use Helm, istio, or etc. And then tech debt makes the decision unchangeable.
Post reply on HN