Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is K8s Too Complicated? 

jmoiron.net

Post reply on HN
Print view View original

101–110 of 171 posts

Re: Is K8s Too Complicated? 

#101

Post by dsnuh »

I feel like managed Kubernetes solutions are a completely different beast from on-prem physical deployments, and that when people talk about how Kubernetes isn't complicated, they are probably running a webapp on GKE or Minikube on a DO droplet, or something similar.

Running Kubernetes on your own private, on-prem infrastructure - integrating services that live outside the cluster, exposing your cluster services, rolling your own storage providers, adding a non-supported LoadBalancer provider, managing the networking policies, etc., etc. - can quickly become an incredibly messy and complex endeavor.

Re: Is K8s Too Complicated? 

#102

Post by merb »

ownagefool wrote: post #15
hacknat wrote: post #5

Right tool for the right job. Is K8s too complicated? For some use cases it is. They probably should do a better job of discouraging certain use cases, but calling their elevator pitch “bullshit” is hyperbolic. There are exceptions to every rule, but a good rule of thumb is cluster size. If you’re managing less than 25 servers than K8s is probably over kill. As you start to creep north of 40 servers K8s really starts…

I'd say it's actually just a convention vs configuration argument. Kubernetes isn't giving you anything you couldn't have already built with configuration management. It just happens to a standard written by a bunch of people with a background in the problem domain. Personally, I think the abstractions are thoughtful and the system isn't really inherently more complicated than what you'll eventually build anyways, bu…

> If you have a single monolith, written and ran by a single team, it doesn't neccessairly get more complex as you throw more servers at it.

In which world does our monolith only connect to a database that is hosted on a single node ;) Most often we have at least 3 database nodes 3 x nodes, 3 redis nodes, whatever. even when we run everything on vms instead of bare metal instances (for which k8s even works) we would still need to manage them and create an insane amount of work to manage/keep them running. in the worst case people would ssh into them

Re: Is K8s Too Complicated? 

#103

Post by KaiserPro »

K8s is a beast, and it has a fairly specific workflow, which for most people is not a good fit.

For example you need to understand that certain workloads dont fit well on the same box (DBs and anything IO/memory sensitive for example)

Then there is the "default" network setup where each node is _statically_ assigned a /24. (because macvtap + dhcp is "unreliable" and inelastic apparently.)

Now, I've heard a lot of talk about you either use k8s or ssh to manage a fleet of machines. That pretty much a wrong comparison.

K8s provides two things a mechanism for shared state (ie. I am service x and I can be found on ips y & z) and a scheduler that places containers on hosts (and manages health checks.)

If your setup has a simple config scheme, (using a simple shared state mechanism, like a DB, or a filesystem, or DNS) or you have no issues with creating highly automated deployments using tools like CF, chef, anisble cloudformation $other, then k8s has vanishing returns (it sure as hell doesn't scale to the 50,000 node count, because its so chatty. )

Basically its a poor man's mainframe, where all the guarantees of a process running correctly regardless of what is happening to the hardware has been taken away.

Re: Is K8s Too Complicated? 

#105

Post by peterwwillis »

K8s does a relatively good job at tackling a very large problem space, but it never tried to tackle the other problem space: learning and using complicated things quickly and easily.

The fact that there is no tool to walk a human through building commonly reproduced configuration files is proof that humans were an afterthought.

Re: Is K8s Too Complicated? 

#106

Post by malkia »

jkaplowitz wrote: post #68
jchw wrote: post #42

Earlier quoted context omitted.

I have read through the nginx ingress controller code in Helm before deploying it into production. What you're saying is pretty much the result of my biggest gripe with Kubernetes, though it's one I don't have a lot of ideas of how to fix; there's too much damn boilerplate. 1000 lines of YAML to store maybe 100 relevant lines. That being said, can you trust that there is not a security vulnerability when you deploy i…

The Kubernetes creators never intended this verbose YAML format to be the long-term format for humans to work with directly. Heptio's ksonnet is where they want to go: https://ksonnet.io No, this is not replacing the YAML under the hood, it's just more convenient for humans as a higher layer.

I found ksonnet, by actually looking for smarter json, jsonnet (which ksonnet is based on). Had little experience, while at Google with borgcfg - and while not the same, it's very similliar in spirit, even has easier to understand evaluation rules (unlike borgcfg, which I could never get fully, or I would understand them when focusing, and then if I haven't used them in a while would completely forget again).

Re: Is K8s Too Complicated? 

#107

Post by inteleng »

pstadler wrote: post #13
hacknat wrote: post #5

Right tool for the right job. Is K8s too complicated? For some use cases it is. They probably should do a better job of discouraging certain use cases, but calling their elevator pitch “bullshit” is hyperbolic. There are exceptions to every rule, but a good rule of thumb is cluster size. If you’re managing less than 25 servers than K8s is probably over kill. As you start to creep north of 40 servers K8s really starts…

Even though I fully agree with you, running a little 3 node cluster just for fun is amazing. Thanks to Rook and an Nginx ingress controller with kube-lego, I’m able to deploy applications leveraging distributed storage and getting tls secured endpoints without a single ssh session. This, in my point of view, is absolutely powerful. Shameless plug, I‘ve been working on a project explaining how to run small scale clust…

> I’m able to deploy applications leveraging distributed storage and getting tls secured endpoints without a single ssh session

Is it grammatically possible to use more buzzwords in a single sentence? Reading that made me want to vomit.

Re: Is K8s Too Complicated? 

#108

Post by eric_h »

ad_hominem wrote: post #81
Filligree wrote: post #38

Earlier quoted context omitted.

You mean, it's stable. That sounds like a good reason to use it, not to avoid it.

Not really; I had an issue that silently broke renewals. When I found the open issue that corresponded to it the maintainers were herding people to cert-manager. There are a lot of issues where they are doing that (besides very obviously at the top of the README with warning symbols). At the very least it's eventually going to break on newer versions of Kubernetes. Maintenance-only != LTS.

Maintenance-only usually means deprecated. There are some areas in the software world where maintenance-only can last a long time and can be de facto LTS. The Kubernetes ecosystem is not one of them.

Re: Is K8s Too Complicated? 

#109

Post by pstadler »

ad_hominem wrote: post #20
pstadler wrote: post #13

Earlier quoted context omitted.

Even though I fully agree with you, running a little 3 node cluster just for fun is amazing. Thanks to Rook and an Nginx ingress controller with kube-lego, I’m able to deploy applications leveraging distributed storage and getting tls secured endpoints without a single ssh session. This, in my point of view, is absolutely powerful. Shameless plug, I‘ve been working on a project explaining how to run small scale clust…

You may want to update your tutorial to use cert-manager because kube-lego is in maintenance-only mode

Thanks for the hint, I didn‘t know that. Related issue: https://github.com/hobby-kube/guide/issues/47

Re: Is K8s Too Complicated? 

#110

Post by devonkim »

majewsky wrote: post #79
ownagefool wrote: post #15

Earlier quoted context omitted.

I'd say it's actually just a convention vs configuration argument. Kubernetes isn't giving you anything you couldn't have already built with configuration management. It just happens to a standard written by a bunch of people with a background in the problem domain. Personally, I think the abstractions are thoughtful and the system isn't really inherently more complicated than what you'll eventually build anyways, bu…

> Kubernetes isn't giving you anything you couldn't have already built with configuration management. Configuration management does not give you loadbalancing. Configuration management does not give you rolling upgrades. I mean, sure, you can do this stuff with CM as well, but with k8s, there's nothing to build. It's already there

And there’s two sides to the equation - even if you have a world class orchestration platform if your application architecture is extremely stateful and can’t tolerate certain services being down the platform can not help. Most places I’ve observed (large and small) simply do not write software in a manner that can be containerized effectively. The abstractions and patterns developers have as a default in most languages tends toward system designs (emergent) where people as a rule write and cache files locally, datastores are expected to stay up forever and never change hostnames, and that service discovery is “too complicated” to ever use.
Post reply on HN