Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nomad vs. Kubernetes

nomadproject.io

Post reply on HN
Print view View original

101–110 of 369 posts

Re: Nomad vs. Kubernetes

#101

Post by 3np »

numbsafari wrote: post #91
hacker_newz wrote: post #83

I've run into so many issues with Nomad that it really doesn't make sense to compare the two. Many are well documented in Github issues and occur when utilizing other Hashicorp products.

I'm curious how responsive you have found them to issues you've identified. Any particular problems you've experienced that have been long standing an unresolved? I occasionally like to kick the tires on Nomad, but so far haven't found it compelling enough to switch away from my current solution, GKE (mostly because of how integrated it is with everything else I am doing and, at this point, familiarity). So, curious…

Not the person you’re replying to, but I’m a couple of years in on Nomad, coming from kubernetes.

I’ve reported and participated in a few issues. Most got fixed within the next couple of minor versions, some went neglected.

I think the somewhat recent 1.0-release was pretty well-timed. There are still a few rough edges (though it’s gotten a lot better continuously over than last year). Specifically, Consul Connect and federation (especially together with ACLs and mTLS) are just about coming together now.

If you don’t rely on Connect and federation across regions, I’d say it’s reliable for production use. Other than that maybe wait 6-12 months and reassess.

I’d also advise to put ACLs and TLS in place before you start putting production loads there; it’s not great to introduce in a running cluster.

Re: Nomad vs. Kubernetes

#103

Post by aequitas »

kakoni wrote: post #36

Also worth noting when running your own k8s(or k3s) cluster; > Running Kubernetes is not free. It is expected that it will take about 0.5 to 0.75 of a single core on an embedded system like a Raspberry Pi just as a baseline. [1] [1] https://github.com/k3s-io/k3s/issues/2278

Ran into this recently with a RPi 3b K3s cluster. Before I could easily allow running workloads on the master node, but since a recent update that node will just get overloaded until unresponsive. Yay for automated hands off updates I guess :).

Re: Nomad vs. Kubernetes

#104

Post by papito »

gilbetron wrote: post #92
tazjin wrote: post #2

As a Xoogler it's always seemed weird to me how Kubernetes was compared to Borg. Kubernetes covers a much larger set of things than Borg alone, and I don't necessarily think that's for the better. Being written in a language that isn't well-suited to large projects and refactoring efforts doesn't help either. Nowadays I don't have use-cases for either, but from playing around with Nomad it felt a lot more "Borg-y" th…

> Being written in a language that isn't well-suited to large projects and refactoring efforts doesn't help either. How is Go not suited to those? I'm not seeing it - and are you comparing Go to Java or C++?

Go is a very hard language to work on as a team. It does not push a lot of conventions on you, and when it does, they just seem weird and anachronistic.

Even the original k8s codebase was notoriously horrible in Go, and that was "from the source".

Ironically, HashiCorp has a better Go codebase, and that's where I picked best practices from, not from Google.

The problem with Go is that the Googlers fail to challenge some of the design decisions. The giants of computer science working on it at the company cannot be questioned, and that just leads to a broken feedback process.

As someone said, Go is the new C, skipping decades of new language design concepts.

Re: Nomad vs. Kubernetes

#105

Post by ledneb »

mkopinsky wrote: post #99

So far I've kept things simple, avoided k8s/HashiStack/etc by using docker compose with a simple docker-compose.yml for each server. This has been working well, but I'm starting to feel the pain points - HA requires copy-pasting yaml, I need to specify which services are on each server, and so on. What's the simplest next step I can take? I'd like something with (close to) the simplicity of docker compose, but ideall…

Assuming serverless is out if the question for your use case, have you tried spending a couple of days investigating a managed Kubernetes cluster with node autoscaling enabled? EKS, AKS, GKE...

Honestly it sounds like you could be at the point where K8s is worthwhile.

Re: Nomad vs. Kubernetes

#106

Post by dikei »

jnsaff2 wrote: post #34

I liked Mesos when I worked on it, and it's been replaced by more modern tools like Nomad, but every time I have to work on k8s it's .. well it's being promoted like a cult, has a cult following and I think the whole thing is set up to suck up complexity and layering. How do you deploy a thing to run on k8s? One would think you deploy a manifest to it and that's it. Like yaml or json or hcl or whatever. No. The built…

> How do you do networking? Layer a service mesh with custom overlay networking that abstract upon k8s clusterip.

Overlay network is easy if you don't need your pod and service IP to be routable from outside the cluster. It took me less than 1 afternoon to learn to use Kubespray to deploy an on-premise cluster with Calico for overlay network.

Re: Nomad vs. Kubernetes

#107

Post by proxysna »

oneweekwonder wrote: post #98
omneity wrote: post #87

Nomad is amazing. We've been using it alongside Consul for close to 2 years at this point at Monitoro.co[0]. We started by using it as a "systemd with a REST API" on a single server, and gradually evolved into multiple clusters with dozens of nodes each. It has been mostly a smooth ride, even if bumpy at moments, but operationally speaking Nomad is closer to Docker Swarm in simplicity and Kubernetes in terms of the f…

> Nomad is closer to Docker Swarm in simplicity and Kubernetes in terms of the feature set. This a question I still need to google but what features does Kubernetes have that Docker Swarm needs? Because the perceived complexity of Kubernetes just blows my mind, where Docker Swarm seems a lot more simpler for the same benefits, but my its just abstracted away? I will say upfront im naive when it comes to container tec…

Swarm has a lot of issues. Some that are on the surface like bad networking and stemming from it scaling issues. Others are related directly to Mirantis. Company that owns docker.inc now. It neglects the swarm part of the docker, even was planning to straight up sunset swarm and move everyone to k8s. They do maintenance, and add feature or two a year which is not enough. Swarm is great for small deployments, as it only requires a docker daemon present. Otherwise you should look towards nomad/k8s/whatewer the cloud solution is.

Re: Nomad vs. Kubernetes

#108

Post by throwaway894345 »

q3k wrote: post #43
jnsaff2 wrote: post #34

I liked Mesos when I worked on it, and it's been replaced by more modern tools like Nomad, but every time I have to work on k8s it's .. well it's being promoted like a cult, has a cult following and I think the whole thing is set up to suck up complexity and layering. How do you deploy a thing to run on k8s? One would think you deploy a manifest to it and that's it. Like yaml or json or hcl or whatever. No. The built…

> How do you deploy a thing to run on k8s? kubectl apply -f ~/git/infra/secretproject/prod.{json,yaml} One JSON/YAML file too uwieldy? Generate it using jsonnet/CUE/dhall/your favourite programming language. Or just talk directly to the Kubernetes API. You don't have to use Helm - in fact, you probably shouldn't be using Helm (as the whole idea of text templating YAML is... thoroughly ignorant in understanding what K…

> kubectl apply -f ~/git/infra/secretproject/prod.{json,yaml}

This is fine for the first deploy, but if you delete a resource from your manifests then kubectl doesn’t try to delete it from the cluster, so in practice you need something like Terraform (or perhaps ArgoCD) which actually tracks state. And of course as you mention, you probably want to generate these manifests to DRY up your YAML so you can actually maintain the thing.

I would love to hear from others how they solve these problems.

> as the whole idea of text templating YAML is... thoroughly ignorant

1000%

Re: Nomad vs. Kubernetes

#109

Post by holografix »

thefounder wrote: post #44
jnsaff2 wrote: post #34

I liked Mesos when I worked on it, and it's been replaced by more modern tools like Nomad, but every time I have to work on k8s it's .. well it's being promoted like a cult, has a cult following and I think the whole thing is set up to suck up complexity and layering. How do you deploy a thing to run on k8s? One would think you deploy a manifest to it and that's it. Like yaml or json or hcl or whatever. No. The built…

>> jeesh. why? If you look at Google they build a whole PaaS (Cloud Run/Appengine) on top of k8 and I guess that's the way it's meant to be used. IaaS -> K8 -> PaaS -> deploy.sh

AppEngine does not run on K8s. Cloud Run _can_ run on GKE with Cloud Run for Anthos or oss kNative

Re: Nomad vs. Kubernetes

#110

Post by tekkk »

_wldu wrote: post #96
tekkk wrote: post #84

Earlier quoted context omitted.

What was wrong with ECS Fargate if you don't mind me asking? Too expensive? Too vendor locked-in?

We want to apply consistent policies and technical controls across multiple dev groups that have multiple AWS accounts. K8s seems to be a good solution for that.

All right, thanks! Just wondered as I have a ECS setup and I really prefer not having to switch unless I have to.
Post reply on HN