Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nomad vs. Kubernetes

nomadproject.io

Post reply on HN
Print view View original

361–369 of 369 posts

Re: Nomad vs. Kubernetes

#361

Post by neop1x »

otabdeveloper4 wrote: post #182
farresito wrote: post #18

Earlier quoted context omitted.

> Go even replaced a low-level language like C++ and achieved the same result in the end Did it, though? Honest question. I get the feeling that it ended up competing with Java (and Python) more than it ended up replacing C++. The C++ folks seem to be way more into Rust than Go.

Yes. The old-school PHP programmers seem to eventually move to Go. Going back to Go from C++ is almost unheard of.

I moved from C++ to Go. It made programming enjoyable to me again. With couple of lines of code, huge and well-thought system libraries and very accessible external modules and gomod I can create a working SW fast. No complicated configuratioms, no cmake, no autoconf, no cryptic compiler errors, no operator overload surprises, no slow template mess, no missing debugging symbols. Of course if you work on C++ rendering engine or you have a huge project already written with all the build and dependency workflow done and it is working fine then Go won't help you or solve anything for you. But I am not going to write anything in C++ ever again (except firmware for microcontrollers / Arduino). It was just too frustrating and too complicated. I can imagine most C++ developers still don't know all language features C++ offer because is simply over-complicated!

Re: Nomad vs. Kubernetes

#362

Post by [deleted] »

_y5hn wrote: post #332
EdwardDiego wrote: post #327

Earlier quoted context omitted.

Go binaries still include the Go runtime which handles GCing etc., much like Python. They're not VMs, but it's really a semantic difference, if you're comparing it to Java, especially now that GraalVM native compilation is gaining popularity.

Sure. Runtime is maybe a better word, though gc and such is not "free".

[deleted]

Re: Nomad vs. Kubernetes

#363

Post by hosh »

throwaway894345 wrote: post #219
hosh wrote: post #174

Earlier quoted context omitted.

Before there was Helm, I wrote my own tool and framework in Ruby (called Matsuri) to generate and manage manifests. I still use it. However, the source of truth is still on the kube api server. I did write in functionality to diff what is in the git repo and what is in the source of truth. There is an additional abstraction that bundles together resources so that a single command can converge them all together. The u…

> Before there was Helm, I wrote my own tool and framework in Ruby (called Matsuri) to generate and manage manifests. I've built these kinds of things too in Python (or Starlark). It's an improvement over YAML, but I often feel the pain of dynamic typing. > The Ruby code can access anything Ruby to generate the manifests, so I have a Turing-complete language and I can use class inheritance, mixins, method overrides.…

I don't really see this tooling as a dynamic configuration language so much as a manifest generator. That means I also have tooling for diffing, debugging, and converging manifests. If you are just focused on configuration, I can see why Turing-completeness will seem like unnessary complexity.

I use classes and mixins to be able to generate similar manifests with slight differences across different clusters or environments. I sometimes use imports (I/O) for manifests provided by an upstream (such as from AWS docs), do transforms to get manifest I want.

I modelled the design off of Chef, which will also declaritively define and converge a set of systems towards desired state. Well-known paths and conventions helps keep things organized.

I havn't had a problem with dynamic typing, but I have also been using Ruby in application development for over 10 years. You might see it as unneeded complexity, but I have been able to use the flexibility for years now. This tooling was designed for a team that uses Ruby as the primary language, and uses designs and idom that would be familiar for a Ruby dev team. It is definitely opinionated and I don't expect it to be universally useful for everyone.

Re: Nomad vs. Kubernetes

#364

Post by ookblah »

politelemon wrote: post #278
ookblah wrote: post #260

i would say the "simplest" setup for us has been ECS fargate (even thought there's some major lock in). Very easy to spin up a cluster and you don't have to manage the underlying structure. If you use docker-compose or equivalent for local dev you just define your tasks/services and launch. Even pulling in ENVs is easy with SSM.

What kind of lock-in? One of the reasons I started using it was due to the minimal lock-in, so we've arrived at different conclusions. A task definition is required, which is sort of a docker-compose.yml analogue, but it's also possible to use docker compose directly with ECS Fargate. So my 'get out of AWS' is pretty simple, I already have the docker-compose.yml ready.

i suppose what i mean is you'd never be able to take a ECS task definition/service and move it over to some other provider. whereas if u did some kubernetes setup u could conceivably do that, moving form AWS to GCP.

Re: Nomad vs. Kubernetes

#365

Post by zie »

je42 wrote: post #277
zie wrote: post #249

Earlier quoted context omitted.

Sure, you pay someone else to keep k8s alive, it's not so bad, but it's expensive to do that. You generally need a full-time team of people to keep a k8s deployment alive if you are running it yourself. I keep Nomad alive part-time. It's operationally simple and easy to wrap ones head around it and understand how it works.

nope. have a GKE cluster running "unattended" for months now. looks fine. ;)

GKE is the industry's first fully managed Kubernetes service...

I.e. you don't run k8s, Google does it for you. I'm talking about where YOU run k8s, not run ON k8s. I agree running ON k8s is pretty easy.

Re: Nomad vs. Kubernetes

#366

Post by zie »

coverj wrote: post #283
sciurus wrote: post #281

Earlier quoted context omitted.

Which cloud providers don't? I'm only familiar with AWS and GCP, and they both have a base hourly charge per-cluster.

DigitalOcean and Linode that I'm aware of, may be others

DigitalOcean charges extra for k8s nodes(compared to VPS), So does Linode last I bothered to check.

Re: Nomad vs. Kubernetes

#367

Post by narism »

still_grokking wrote: post #335
m00x wrote: post #308

Earlier quoted context omitted.

K8s itself is divided in multiple parts, where you can customize to your own liking, and you can swap parts if you'd like as long as the APIs are similar. It's very much built the UNIX way.

Where can I find those alternative elements that can be swapped? If this is true there should be a lot of them, right?

If your complaint about Kubernetes is that it doesn’t provide you enough choice/extensibility, you’re probably not looking hard enough.

Runtimes (CRI) - Docker shim (deprecated), containerd, CRI-O, kata

Networking (CNI) - Flannel, Calico, Cilium, cloud specific ones, many more

Storage (CSI) - way too many device plugins - GPU, TPU, RDMA/SRIOV NICs

Data store - etcd, dqlite in microk8s, SQLite/postgresql/mysql in k3s

DNS - kube-dns (somewhat deprecated), CoreDNS

Ingress - NGINX (multiple), HAProxy, Envoy (many), etc

Kubelet -https://github.com/virtual-kubelet/virtual-kubelet

Kube-proxy - Cilium can act as a replacement

Cloud-controller-manager for each cloud provider

kube-scheduler - https://kubernetes.io/docs/tasks/extend-kubernetes/configure...

kube-apiserver and kube-controller-manager are two parts where I’m not aware of any other implementations but a) they are kind of the heart of k8s and b) can be easily extended with CRDs/operators.

Re: Nomad vs. Kubernetes

#368

Post by moljac024 »

p_l wrote: post #200
clvx wrote: post #45

Earlier quoted context omitted.

The worst part of k8s is definitely its configuration system. I don’t like helm as it’s combining packaging, templating and deployment lifecycle. I feel like each of this components should’ve been its own abstraction.

So just don't use Helm. It's not a core, crucial, or required part of k8s, it's just a reasonably popular related project... That I usually ban from environments before the rot can take root.

what are the alternatives?

Re: Nomad vs. Kubernetes

#369

Post by marvinblum »

GordonS wrote: post #352
marvinblum wrote: post #33

I've been running a production-grade Nomad cluster on Hetzner for the past 1 1/2 years and it's fantastic. It was amazingly easy to set up compared to Kubernetes (which I also did), the UI is awesome, updates haven't broken anything yet (as long as you follow the changelog) and it's stable. I really like the separation of concerns the HashiStack offers. You can start out just using Consul for your service meshing, an…

I'm really curious about the local development story for Nomad. I'm using Docker Swarm in production, which let's me easily spin up the same services locally as I do for production - it's really nice to have one set of yaml files that can be used across all environments, and of course it gives more confidence before shipping to prod. How do you handle local development? Do you have Docker Compose files specifically f…

Sorry, didn't see your reply because... well HN has no notification thing I think? Anyways, we don't run Nomad locally. All of our services can be started locally without additional tools other than Go.
Post reply on HN