Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is K8s Too Complicated? 

jmoiron.net

Post reply on HN
Print view View original

131–140 of 171 posts

Re: Is K8s Too Complicated? 

#131

Post by lowbloodsugar »

pstadler wrote: post #124
lowbloodsugar wrote: post #123

Earlier quoted context omitted.

You: Look what I did with k8s Top voted reply: You had to use a thing to do what you want, and that thing isn't even supported any more. [Either I was hallucinating at the time and imagined it, or there was a reply that got deleted] I think you've just quite nicely demonstrated the original authors point.

You sound like you‘re new to this business.

Sometimes the people "new to this business" have the best ideas. It's more likely that I'm a grumpy old man worried about his lawn.

Re: Is K8s Too Complicated? 

#132

Post by AndyNemmity »

markbnj wrote: post #129
AndyNemmity wrote: post #119

Earlier quoted context omitted.

The stable chart thing is so weird. Internally use we some abstractions, but I looks at stable charts and it requires so much time just to understand all of what's going on. Everything is a variable pointed to values, and you can't reason about any of it. It seems like the hope is, just ignore it all, and the docs are good, and just follow them, but I don't live in any kind of world I can do that. And the commits, an…

> It seems like the hope is, just ignore it all, and the docs are good, and just follow them, but I don't live in any kind of world I can do that. Yep, agreed, we've used very few charts from stable, and in some cases where we have we needed to fork and change them, which is its own special form of suck. The one I contributed was relatively straightforward: a deployment, service and a configMap to parameterize and mo…

Right, I'm in particular working on charts for ELK, and it's just a mess. I just took down all my data (in staging, so all good) due to a PVC. The charts won't update without deleting them when particular parts of the chart change, but if you delete them, you lose your PVC data.

So I find the note in an issue somewhere stating, this is.. intentional?.. and that of course you need some annotation that will change it.

Let alone the number of things like, xpack, plugins, the fact that java caches the DNS so endpoints don't work on logstash, on and on.

It seems like everyone is saying operators are going to be the magical way to solve this, but if anything it seems like one set of codified values, that don't address any of the complexity.

Re: Is K8s Too Complicated? 

#133

Post by justinsaccount »

zie wrote: post #126
justinsaccount wrote: post #117

Earlier quoted context omitted.

I don't really see how nomad is operationally simpler than k8s. To run a service behind something like traefik on k8s I would: bootstrap a CA for cluster tls run etcd run k8s (apiserver,etc,kubelets) run your application run the traefik ingress controller To run a service behind something like traefik on nomad you would: bootstrap a CA for cluster tls run consul run nomad (servers, clients) run your application run t…

We are talking about different things. I'm talking about keeping Kubernetes/Nomad alive and breathing and happy. The ops part of devops. You are talking about running stuff under them. I agree they are similar in running applications under them. Operationally simple: * 1 binary, for both servers and agents. * 1 config file. For consul & nomad 2 config files. * Upgrades are simple, bring down a node, replace binary, s…

No, I'm talking about operationally too. If your consul and nomad deployments are only one binary and one config file then you're not using TLS . Half the effort of setting k8s up is bootstrapping the CA and certs for etcd and the k8s components

> k8s is a very complex beast, with many, many binaries and configs

Because it's much more in the unix toolset philosophy, let 1 tool do 1 thing well. Is that a bad thing now?

hyperkube does put all the server side components in a single binary, there's still a bit of configuration though. A lot of the options are repetitive, I bet one could wrap hyperkube with a single config file and some defaults and the end result would look a like like nomad.

Re: Is K8s Too Complicated? 

#134

Post by zie »

justinsaccount wrote: post #133
zie wrote: post #126

Earlier quoted context omitted.

We are talking about different things. I'm talking about keeping Kubernetes/Nomad alive and breathing and happy. The ops part of devops. You are talking about running stuff under them. I agree they are similar in running applications under them. Operationally simple: * 1 binary, for both servers and agents. * 1 config file. For consul & nomad 2 config files. * Upgrades are simple, bring down a node, replace binary, s…

No, I'm talking about operationally too. If your consul and nomad deployments are only one binary and one config file then you're not using TLS . Half the effort of setting k8s up is bootstrapping the CA and certs for etcd and the k8s components > k8s is a very complex beast, with many, many binaries and configs Because it's much more in the unix toolset philosophy, let 1 tool do 1 thing well. Is that a bad thing now…

You keep going on about setting k8s up, and not about maintenance. How much time in a week do you take to babysit your k8s cluster? Do you have an HA setup?

OK TLS takes 3 files, 2 for the key and crt and 1 for the config. If you get your TLS certs out of the vault PKI backend, it's very, very simple (https://www.vaultproject.io/docs/secrets/pki/index.html) the linked page covers the complete steps.

Again, I keep talking about maintaining Nomad/k8s for years. I've been running nomad in production for a few years now, I've had no downtime from nomad, and I spend about an hour doing upgrades every once in a while. I don't worry about nomad, it's just there and works. I run 3 nomad servers per data center for an HA setup. k8s doesn't even test their HA setup in development (source: https://kubernetes.io/docs/admin/high-availability/building/) . There is no way it works out well in real life, if they don't even test it yet.

Nobody I know that runs k8s pretends it's easy to keep running for years. Most places that run k8s have dedicated engineers to babysit k8s. I babysit our nomad, and lots of other infrastructure, and I do development of applications as well.

Re: Is K8s Too Complicated? 

#135

Post by tomsthumb »

cygned wrote: post #16

I started with k8s beginning this year and from my point of view, the documentation is not good - and a major pain point when trying to get started. Each part on its own is good and well written, but it lacks the overall picture and does not connect pieces well enough. For example, the schema definitions for the all the configuration files are not linked from the official docs (at least I wasn’t able to find them). T…

The kubernetes up and running book gives a better big picture view than the documentation online does. It does a thorough job of developing motivation and context for using a broad swath of the system in a cohesive way.

Re: Is K8s Too Complicated? 

#136

Post by StavrosK »

vectorpush wrote: post #111
inteleng wrote: post #107

Earlier quoted context omitted.

> I’m able to deploy applications leveraging distributed storage and getting tls secured endpoints without a single ssh session Is it grammatically possible to use more buzzwords in a single sentence? Reading that made me want to vomit.

The words have meaning. If you don't understand them you could just ask for them to be explained rather than throwing out insults. I suspect you actually do understand the meaning of what was written though so I suppose that means you're just trying to start a flamewar.

The problem is that "leveraging" can be replaced with "using" every time, with the added benefit that it won't leave a bad taste in the reader's mouth.

Re: Is K8s Too Complicated? 

#137

Post by Axsuul »

akavel wrote: post #114
hacknat wrote: post #5

Right tool for the right job. Is K8s too complicated? For some use cases it is. They probably should do a better job of discouraging certain use cases, but calling their elevator pitch “bullshit” is hyperbolic. There are exceptions to every rule, but a good rule of thumb is cluster size. If you’re managing less than 25 servers than K8s is probably over kill. As you start to creep north of 40 servers K8s really starts…

What good alternative can you suggest for less than 25 servers?

Docker Swarm

Re: Is K8s Too Complicated? 

#138

Post by markbnj »

AndyNemmity wrote: post #132
markbnj wrote: post #129

Earlier quoted context omitted.

> It seems like the hope is, just ignore it all, and the docs are good, and just follow them, but I don't live in any kind of world I can do that. Yep, agreed, we've used very few charts from stable, and in some cases where we have we needed to fork and change them, which is its own special form of suck. The one I contributed was relatively straightforward: a deployment, service and a configMap to parameterize and mo…

Right, I'm in particular working on charts for ELK, and it's just a mess. I just took down all my data (in staging, so all good) due to a PVC. The charts won't update without deleting them when particular parts of the chart change, but if you delete them, you lose your PVC data. So I find the note in an issue somewhere stating, this is.. intentional?.. and that of course you need some annotation that will change it.…

You're using a statefulset? Here's a tip: you can delete a statefulset without deleting the pods with `kubectl delete statefulset mystatefulset --cascade=false`. The pods will remain running, but will no longer be managed by a controller. You can then alter and recreate the statefulset and as long as the selector still selects those pods the new statefulset will adopt them. If you then need to update the pods you can delete them one at a time without disturbing the persistent volume claims, and the controller will recreate them.

Re: Is K8s Too Complicated? 

#139

Post by justinsaccount »

zie wrote: post #134
justinsaccount wrote: post #133

Earlier quoted context omitted.

No, I'm talking about operationally too. If your consul and nomad deployments are only one binary and one config file then you're not using TLS . Half the effort of setting k8s up is bootstrapping the CA and certs for etcd and the k8s components > k8s is a very complex beast, with many, many binaries and configs Because it's much more in the unix toolset philosophy, let 1 tool do 1 thing well. Is that a bad thing now…

You keep going on about setting k8s up, and not about maintenance. How much time in a week do you take to babysit your k8s cluster? Do you have an HA setup? OK TLS takes 3 files, 2 for the key and crt and 1 for the config. If you get your TLS certs out of the vault PKI backend, it's very, very simple ( https://www.vaultproject.io/docs/secrets/pki/index.html ) the linked page covers the complete steps. Again, I keep t…

> How much time in a week do you take to babysit your k8s cluster? Do you have an HA setup?

I don't have a k8s cluster... so zero :-)

I don't have a nomad cluster either, because every time I look at it and start planning out what I would need to do to bootstrap consul+nomad and secure it, it starts to look more like a k8s install.

> There is no way it works out well in real life,

except that every cluster on GKE or created using kops, kubespray, or even kubernetes the hard way is HA, so it's not like no one is running an HA cluster. I think from k8s point of view, there isn't much to test as etcd is doing all the work.

Re: Is K8s Too Complicated? 

#140

Post by justinsaccount »

lifeisstillgood wrote: post #28

Side note - i stumbled across an attempt to rebuild K8 from scratch in python (it looked like a learning attempt) - but i forgot to bookmark it - anyone seen something similar

Are you sure it was k8s, and not maybe docker?

https://github.com/tonybaloney/mocker

Post reply on HN