11–20 of 449 posts
Googler, opinions are my own. Direct link to the docs: https://cloud.google.com/kubernetes-engine/docs/concepts/aut... Google has a tool by the same name (autopilot) internally that does close to the same thing. There was a paper published on it 10 months ago talked about here: https://news.ycombinator.com/item?id=22980467
Containers add vast complexity, add another layer of complexity on top.
- I can store anything in a secret? Let's have thousands of cat images. Etcd then stops working because we have over 2GB of funny cats in the key store.
- I can run a root Pod? Lets mount the docker socket and start building images with it. Oh and by the way, I never clean those up and my Node simply fills up. Also I add some additional docker networks that break Pod to Pod networks.
- Istio is nice - why we don't add automatic injection for Pods in all namespaces? Including kube-system? And then they brick kube-proxy and the cluster stops working.
- I can use validating webhooks for better security? Lets watch on all resources. To keep it more secure lets set the failure policy of the webhook to Fail, so we never admit any modification without the apiserver to make a call to out webhook. Whats that? My single replica webhook has was evicted from the Pod (we didn't add any resource requests and limits) and now it cannot even be created or scheduled because kube-controller-manager and kube-scheduler cannot update their lease and they lost leadership and now are idling, effectively bricking the entire cluster.
Google would reduce the pain points with this change, however they would still face countless other issues with Kubernetes.
Sadly, this is a typical Register headline. Google did not say "Kubernetes is too complex" but rather, they are making this new tool - called Autopilot - that is an abstraction layer on top of Kubernetes for certain types of applications / companies. This Autopilot system still uses Kubernetes AFAICT.
This was glaringly obvious from day one. Containers add vast complexity, add another layer of complexity on top.
I think the single biggest mistake people make with Kubernetes is implementing it too soon. Last company I worked for spent piles of time fighting K8s when a simple well implemented cluster would have done the job. It makes a lot of sense to build portable infrastructure, but you can scale a long ways with much simpler technologies.
Build debs, use the package manager. VMs are already an artificial abstraction, just use them.
Way too complex for my tastes, but if you're convinced you need a highly automated control plane for those things then pushing .deb or .rpm packages doesn't even come close to solving that problem.
As usual it comes down to how you define the problem. You can't dissuade people from using k8s by comparing and contrasting k8s to alternatives; you've already ceded the debate over how to define the problem at that point. W'ever its relative merits, k8s is a reasonable approach to the problem of automating the control plane for "scaleable" services.
I think the single biggest mistake people make with Kubernetes is implementing it too soon. Last company I worked for spent piles of time fighting K8s when a simple well implemented cluster would have done the job. It makes a lot of sense to build portable infrastructure, but you can scale a long ways with much simpler technologies.
It's an insanely complex solution to a very niche problem - scaling stateless web app backend nodes written in scripting languages.
Stray even a little bit off the garden path and you start feeling pain.
I can't fight the feeling that this is all circling back to the application server stuff that was popular for a time. And, really, I can't tell when those went so wrong. :(
1. Someone has an idea. It's alright. Really good for their use case. Someone else hears about it, likes it, and adapts it to a similar use case. So and so forth until the idea has a large user base
2. Employees of large companies hear about the idea and implement it
3. Marketing gets a hold of the idea, gives it a flashy name, and uses it in promotions
4. A majority of the loudest voices in the industry get on board so everyone starts forcing this idea on every imaginable use case
5. A lot of people realize this is all too much extra work without much benefit so they start looking for more appropriate solutions
6. return to 1
It's happened with mainframes, object oriented programming, services, micro-services, web-all-the-things, blockchain, steaming data (kafka), sql, nosql, containers, agile, VMs, cloud, and many other things.
It's just technology people are the worst at getting caught up today's fad and I wish we could try to not do that as much.
From the TA: The maximum number of pods per node is 32, as opposed to 110 on standard GKE. This is sad.. I don't understand why providers do it. It makes it very expensive to run small staging clusters. There is some reference on the current state here: https://docs.google.com/spreadsheets/u/0/d/1yhkuBJBY2iO2Ax5F...
The pricing is... not cheap: https://cloud.google.com/kubernetes-engine/pricing
Comparable-ish with Fargate. You probably wouldn't be using this with an eye to save money unless you think (or have measured) that you'd spend more on operations, security or compliance otherwise.
This was glaringly obvious from day one. Containers add vast complexity, add another layer of complexity on top.