Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is K8s Too Complicated? 

jmoiron.net

Post reply on HN
Print view View original

111–120 of 171 posts

Re: Is K8s Too Complicated? 

#111

Post by vectorpush »

inteleng wrote: post #107
pstadler wrote: post #13

Earlier quoted context omitted.

Even though I fully agree with you, running a little 3 node cluster just for fun is amazing. Thanks to Rook and an Nginx ingress controller with kube-lego, I’m able to deploy applications leveraging distributed storage and getting tls secured endpoints without a single ssh session. This, in my point of view, is absolutely powerful. Shameless plug, I‘ve been working on a project explaining how to run small scale clust…

> I’m able to deploy applications leveraging distributed storage and getting tls secured endpoints without a single ssh session Is it grammatically possible to use more buzzwords in a single sentence? Reading that made me want to vomit.

The words have meaning. If you don't understand them you could just ask for them to be explained rather than throwing out insults. I suspect you actually do understand the meaning of what was written though so I suppose that means you're just trying to start a flamewar.

Re: Is K8s Too Complicated? 

#112

Post by SEJeff »

majewsky wrote: post #79
ownagefool wrote: post #15

Earlier quoted context omitted.

I'd say it's actually just a convention vs configuration argument. Kubernetes isn't giving you anything you couldn't have already built with configuration management. It just happens to a standard written by a bunch of people with a background in the problem domain. Personally, I think the abstractions are thoughtful and the system isn't really inherently more complicated than what you'll eventually build anyways, bu…

> Kubernetes isn't giving you anything you couldn't have already built with configuration management. Configuration management does not give you loadbalancing. Configuration management does not give you rolling upgrades. I mean, sure, you can do this stuff with CM as well, but with k8s, there's nothing to build. It's already there

Configuration management absolutely can do this. Note that I'm a builder of kubernetes clusters (on-premise) but also have contributed 400+ patches to the salt configuration management tool.

Salt has an "orchestrate layer"[1] which allows running states on sets of minions. One of those layers can be to configure a service, and another can be to update the load balancers when said service is healthchecking green. Saying these things simply can't be done with configuration management is utterly false. Kubernetes just makes it easier and more approachable for those less skilled in under the covers systems and infrastructure stuff. Kubernetes is a tool that allows you to build things quickly, but it isn't for everyone.

[1] https://docs.saltstack.com/en/2017.7/topics/orchestrate/orch...

Re: Is K8s Too Complicated? 

#113

Post by jacques_chester »

matthewmacleod wrote: post #57

I suppose we have to assume that we should use the right tool for the right job, and all that. And I'm sure that the Kubernetes folk know what they're doing. But I definitely think that it's too complicated without a cutting-edge Kubernetes expert in place to manage it. And even then, it's just a building block for a larger system. I've tried maybe half a dozen times to get started for relatively small workloads – le…

> I'd love something like Heroku that I could run on my own bare metal, or on AWS – the various solutions I've tried were all lacking.

I've worked on or near to Cloud Foundry for about 4 years now, and Heroku was one of the inspirations for it, along with Borg.

Is there something in particular it didn't do well enough?

Re: Is K8s Too Complicated? 

#114

Post by akavel »

hacknat wrote: post #5

Right tool for the right job. Is K8s too complicated? For some use cases it is. They probably should do a better job of discouraging certain use cases, but calling their elevator pitch “bullshit” is hyperbolic. There are exceptions to every rule, but a good rule of thumb is cluster size. If you’re managing less than 25 servers than K8s is probably over kill. As you start to creep north of 40 servers K8s really starts…

What good alternative can you suggest for less than 25 servers?

Re: Is K8s Too Complicated? 

#115

Post by nvarsj »

jaequery wrote: post #10

what are some of the downsides of K8? any horror stories?

(I've been using k8s for several years now, and have a couple dozen contributions so far)

Two big drawbacks.

First, with k8s itself is _stability_. K8s was a design-by-committee attempt to make Borg 2.0 with open source. It had massive feature development (which continues), but little actual real usage of those features. So it has taken a long time for things to get into a good shape. Things are much much better now, because of the popularity. "Given enough eyeballs, all bugs are shallow". But expect to debug things, especially if you use a feature not used by GKE.

The other big downside is it takes a lot of domain knowledge to manage it, and keep up to date (there is no LTS version). There isn't a standard way of deploying it into on-prem either. So that stuff needs to be built.

Re: Is K8s Too Complicated? 

#117

Post by justinsaccount »

zie wrote: post #93
justinsaccount wrote: post #84

Earlier quoted context omitted.

> You can do load balancing and all the other things k8s does Nomad can do many of the things that k8s does, and even some things that k8s can't, but it does not do all the things that k8s does. For example, nomad doesn't have network policies. Also, just to approach the base level of functionality and security in k8s you need to setup consul as well and bootstrap all the tls stuff, and at that point it's not much mo…

Yes, True enough. Arguably, network policies are outside of Nomad's scope. It's a resource scheduler. Nomad doesn't turn up interfaces for you, or do routing of IP traffic. Nomad is not a one-stop shop, like k8s tries to be, it does resource scheduling in a nice declarative manner, that's about it. It's much more in the unix toolset philosophy, let 1 tool do 1 thing well, and make integration as painless as possible.…

I don't really see how nomad is operationally simpler than k8s. To run a service behind something like traefik on k8s I would:

  bootstrap a CA for cluster tls
  run etcd
  run k8s (apiserver,etc,kubelets)
  run your application
  run the traefik ingress controller
To run a service behind something like traefik on nomad you would:

  bootstrap a CA for cluster tls
  run consul
  run nomad (servers, clients)
  run your application
  run traefik somehow
I think the only thing that is really more complicated in k8s is the networking stuff, but that's only because it has more features, like having cluster dns automatically configured and being able to give every pod its own ip address which means every service can bind to port 80 without conflicts, and policy enforcement.

Re: Is K8s Too Complicated? 

#118

Post by jacques_chester »

perlgeek wrote: post #33

Suppose I'm an application developer who is only interested in infrastructure because there needs to be some to run my stuff. In this scenario, would I actually learn Kubernetes, or would it make more sense to go to straight to a PaaS solution? Like OpenShift (which uses k8s in the backend, I believe), or Cloud Foundry, Stackato etc. I always get the impression that k8s has a lot of good ideas, but doesn't provide ev…

My advice is to go directly to a PaaS. I work for Pivotal R&D in and around Cloud Foundry, so that's my personal horse. But I'd rather that you used any PaaS -- Cloud Foundry, OpenShift, Rancher -- than roll your own.

Building a platform is hard. It's really really hard. Kubernetes commoditises some of the hard bits. The community around it will progressively commoditise other aspects in time.

But PaaSes already exist, already work and either already base themselves on Kubernetes or have a roadmap to doing so.

To repeat myself: building PaaSes is hard. Hard hard hard. Collectively, Pivotal, IBM, SAP and SUSE have allocated hundreds of engineers in dozens of teams to work on Cloud Foundry. We've been at it non-stop for nearly 5 years. Pivotal spends quite literally millions of dollars per year testing the everliving daylights out of every part of it [0][1]. (Shout out to Concourse here)

I fully expect Red Hat can say the same for OpenShift.

Really. Use a PaaS.

[0] https://content.pivotal.io/blog/250k-containers-in-productio...

[1] https://content.pivotal.io/blog/you-deserve-a-continuously-i...

Re: Is K8s Too Complicated? 

#119

Post by AndyNemmity »

markbnj wrote: post #70
jchw wrote: post #42

Earlier quoted context omitted.

I have read through the nginx ingress controller code in Helm before deploying it into production. What you're saying is pretty much the result of my biggest gripe with Kubernetes, though it's one I don't have a lot of ideas of how to fix; there's too much damn boilerplate. 1000 lines of YAML to store maybe 100 relevant lines. That being said, can you trust that there is not a security vulnerability when you deploy i…

> What you're saying is pretty much the result of my biggest gripe with Kubernetes, though it's one I don't have a lot of ideas of how to fix; there's too much damn boilerplate. 1000 lines of YAML to store maybe 100 relevant lines. I think that's more a helm issue than a k8s issue. I've been using helm in production for over a year and k8s for almost three years. Prior to adopting helm we rolled our own yaml template…

The stable chart thing is so weird. Internally use we some abstractions, but I looks at stable charts and it requires so much time just to understand all of what's going on. Everything is a variable pointed to values, and you can't reason about any of it.

It seems like the hope is, just ignore it all, and the docs are good, and just follow them, but I don't live in any kind of world I can do that.

And the commits, and the direction of all of them seem to go more and more impossible to read conditionally rendered symbols.

I've had such a challenge understanding and using helm well enough. Small gotchas everywhere that can just eat up tons of time. This doesn't feel like the end state to me.

Re: Is K8s Too Complicated? 

#120

Post by AndyNemmity »

fear91 wrote: post #24
nimbius wrote: post #12

disclosure: im an engine mechanic with a lot of interest in Linux as a hobby. Kubernetes seems like a great alternative to stuff like openstack that seems like it requires an entire datacenter to get going properly, but I feel like the hype (k8s? really?) is outliving the reality. youre also bucking up against a problem where on smaller scales, it just seems easier to use something else. Maybe not "agile" and all tha…

OpenStack is used mostly for the layer beneath Kubernetes. We use it to provision the VM's/networking for the Kubernetes cluster itself. Kubernetes might not make sense for your small scale projects but it's great when used for enterprise scale microservices. It makes deployments easier, faster and more secure (with things like network policies, namespacing and RBAC).

We run Kubernetes under Openstack, which can deploy hosted Kubernetes. Lots of variations.
Post reply on HN