Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nomad vs. Kubernetes

nomadproject.io

Post reply on HN
Print view View original

281–290 of 369 posts

Re: Nomad vs. Kubernetes

#281

Post by sciurus »

llama052 wrote: post #273
zie wrote: post #249

Earlier quoted context omitted.

Sure, you pay someone else to keep k8s alive, it's not so bad, but it's expensive to do that. You generally need a full-time team of people to keep a k8s deployment alive if you are running it yourself. I keep Nomad alive part-time. It's operationally simple and easy to wrap ones head around it and understand how it works.

A lot of the cloud providers don't bill directly for kubernetes management, instead it's just the node resources. Either way, as another comment points out, Rancher and many other solutions make the orchestration of creating your own Kubernetes cluster really boring. We run a few kubernetes clusters on premise, and for the longest time it was just 1 person running some Kubernetes clusters.. we even have other teams i…

Which cloud providers don't? I'm only familiar with AWS and GCP, and they both have a base hourly charge per-cluster.

Re: Nomad vs. Kubernetes

#282

Post by richwater »

jeswin wrote: post #94

Despite its reputation, Kubernetes is actually quite easy to master for simple use cases. And affordable enough for more complex ones. The fundamental abstractions are as simple as they can be, representing concepts that you'd already be familiar with in a datacenter environment. A cluster has nodes (machines), and you can run multiple pods (which is the smallest deployable unit on the cluster) on each node. A pod ru…

Running a production ready, high availability Kubernetes cluster with proper authn, authz, and resource controls is about the farthest thing from "simple" that I can imagine.

Re: Nomad vs. Kubernetes

#283

Post by coverj »

sciurus wrote: post #281
llama052 wrote: post #273

Earlier quoted context omitted.

A lot of the cloud providers don't bill directly for kubernetes management, instead it's just the node resources. Either way, as another comment points out, Rancher and many other solutions make the orchestration of creating your own Kubernetes cluster really boring. We run a few kubernetes clusters on premise, and for the longest time it was just 1 person running some Kubernetes clusters.. we even have other teams i…

Which cloud providers don't? I'm only familiar with AWS and GCP, and they both have a base hourly charge per-cluster.

DigitalOcean and Linode that I'm aware of, may be others

Re: Nomad vs. Kubernetes

#284

Post by cfigueir »

jeswin wrote: post #94

Despite its reputation, Kubernetes is actually quite easy to master for simple use cases. And affordable enough for more complex ones. The fundamental abstractions are as simple as they can be, representing concepts that you'd already be familiar with in a datacenter environment. A cluster has nodes (machines), and you can run multiple pods (which is the smallest deployable unit on the cluster) on each node. A pod ru…

The comparison is a bit misleading. As someone that has used both nomad and k8s at scale --

- Nomad is a scheduler. Clean and focused. It is very fast. I was an early user and encountered a number of bumps, but that's software. The people at Hashicorp are super sharp and lovely.

- K8s is a lot more. It includes a scheduler, but in the simplest sense, it is a complete control-plane based on the control-loop pattern. You have an API, a scheduler, a db, various controllers, etc. Forget for a moment that most people use it to orchestrate containers -- it's really designed to orchestrate anything. Its API is extensible, you can add and compose controllers -- there are many possibilities once you wrap your head around it.

This is all opinionated and includes a lot of capability. It's just very different.

You can stitch together nomad, consul, vault, and various glue to create a container orchestration system... but when you start wanting to manage the control-planes as though they are the "kind" (the container for example) with meta-control-planes, and you start wanting to orchestrate network, storage, and other dependencies... all while doing this in a multi-tenant environment, then things get interesting.

-charles.

Re: Nomad vs. Kubernetes

#286

Post by ldehaan »

websap wrote: post #238
jeswin wrote: post #94

Despite its reputation, Kubernetes is actually quite easy to master for simple use cases. And affordable enough for more complex ones. The fundamental abstractions are as simple as they can be, representing concepts that you'd already be familiar with in a datacenter environment. A cluster has nodes (machines), and you can run multiple pods (which is the smallest deployable unit on the cluster) on each node. A pod ru…

I couldn't disagree with this more. I manage and run Kubernetes clusters as part of my job, and I can tell you that configuring, running and installing these clusters is no small feat. I don't know much about Nomad, but I would highly encourage most users to not think K8S is simple by any standard. Multiple cloud providers now provide ways to run your code directly, or your containers directly. Unless you have a reas…

I dunno man, k8s is pretty simple, but only if you make it so, I have built (early on) complicated setups with all kinds of functionality, but the complexity was rarely used. I now manage clusters and deployments on those clusters with code, terraform to be exact (tcl expect for instances when terraform can't hang yet) I built a gui interface for terraform to manage deployments of kubernetes on various clouds and bare metal and to manage the deployments To kubernetes. \n Maybe one day this will get too complex and we'll do something else but so far so good.

Re: Nomad vs. Kubernetes

#287

Post by EdwardDiego »

cpuguy83 wrote: post #262
EdwardDiego wrote: post #54

Earlier quoted context omitted.

I just wish klog would go die in a fire, I'm so confused at why logging is so terrible in Go.

klog is the descendant of glog which is a go implementation of Google's logging. Go has nothing to do with it except it's the logging that k8s took on (originally it used glog). -- edit typo

Sorry was referring more to the many different incompatible logging libraries in play.

Also the inability in the ones I've used for the person running the app to set a particular logger to a desired level easily.

I think klog can do this with the vmodules flag, if, and only if, the devs used klog.V() for their logging statements.

Logrus requires the dev to allow the user to configure the log level, common idiom I've encountered is doing so via an env var, but IIRC, that applies to all logging in the app, no way to limit it to particular files/modules.

It's been a real pain at times for me. Feels like the Go philosophy on logging focuses more on the dev controlling it, than the person running it.

Re: Nomad vs. Kubernetes

#288

Post by EdwardDiego »

GlennS wrote: post #66
EdwardDiego wrote: post #54

Earlier quoted context omitted.

I just wish klog would go die in a fire, I'm so confused at why logging is so terrible in Go.

Can you expand on this? My experience with Go has just been making small changes to small programs. So, I don't know what the normal experience is. My experience with logging varies from: print (works fine I guess) import logging (this is pretty good - means I don't have to parse my logs before deciding where to send them) import slf4j (6 logging frameworks and a logging framework disintermediation framework)

I elaborated a bit in my reply above, but basically, multiple logging frameworks with incompatible APIs, few of which offer the fine-grained control the person running the app might need.

But I'm used to the JVM world. And when I first met slf4j, I was like, wtf is this crap, but I appreciate it now as the embodiment of the desire to standardise logging across the Java ecosystem.

While using slf4j does make it trivial to swap out logging frameworks in an app, in my 13 years at my last job, we only did that once, from log4j to Logback, so that's not so important.

But yeah, what I miss from Java land in Go logging is the common approach - loggers and appenders are (usually) configured outside of code, the user can provide their own configuration at runtime to override the config shipped in the jar to troubleshoot issues - especially when you can configure the logging lib to check the conf file every X seconds for changes - allows you to change logger levels on the fly without restarting the app (ditto Logback's JMX configurator).

And lastly, no matter the logging library, configuring them is near identical.

Re: Nomad vs. Kubernetes

#289

Post by welkie »

throwaway894345 wrote: post #216
JelteF wrote: post #187

Earlier quoted context omitted.

I've been using the following command to great effect: kubectl apply --kustomize manifests/ --prune --selector app=myapp It cleans up old stuff from the cluster and also allows you to split your manifests across multiple files.

Whoa. I never knew about --prune.

Yeah... I'm not really sure where this idea of not being able to use the tools available out of the box to deploy apps and do networking comes from. I deploy YAML. If I feel like my YAML is too big, I DRY it using kustomize. I can use --prune if I'm worried about stuff sticking around in the cluster. For networking, I... don't do anything? We get DNS built in. Just use the service name. What else is there to do?

Re: Nomad vs. Kubernetes

#290

Post by still_grokking »

gizdan wrote: post #191
wg0 wrote: post #179

Earlier quoted context omitted.

One of the things that I don't like about Nomad is HCL. It is a language that is mainly limited to HashiCorp tools and there's no wider adoption outside at least not to my knowledge. From the documentation: > Nomad HCL is parsed in the command line and sent to Nomad in JSON format via the HTTP API. So why not just JSON or even JSON at all and not MsgPack or just straight up HCL because that's over and over introduced…

JSON was designed for machine readability, HCL was designed for human readability. HCL requires a lot more code to parse and many more resources to keep in memory vs JSON. I think it completely makes sense to do it this way. K8s is the same. On the server it does everything in JSON. Your YAML gets converted prior to sending to K8s.

Parsing JSON Is a Minefield (2016):

https://news.ycombinator.com/item?id=28826600

Post reply on HN