Google admits Kubernetes container tech is too complex

K8s isn't too complex for what it does and can do.

it's just that a lot of people think they have to use k8s because it's trendy even it doesn't fit at all their needs and scale. Then they whine it's too complex.

classic PEBCAK issue.

And scam retail. I worship Scott Locklin! Savior of Truth!

Thanks, that's really interesting. Everyone has different challenges and requirements, and of course different experiences.

For smaller setups (say 1-10 services) I'm quite happy with cloud config and one VM per process behind one load balancer per service. It's simple to set up, scale and reproduce. This setup doesn't autoscale, but I've never really felt the need. We use Go and deploy one static binary per service at work with minimal dependencies so docker has never been very interesting. We could redeploy almost all the services we run within minutes if required with no data loss, so that bit feels similar to K8s I imagine.

For even smaller companies (many services at many companies) a single reliable server per service is often fine - it depends of course on things like uptime requirements for that service but not everything is of critical importance and sometimes uptime can be higher with a single untouched service.

I think what I'd worry about with a k8s config which affects live deployments is that I could make a tweak which seemed reasonable in isolation but broke things in inscrutable ways - many outages at big companies seem to be related to config changes nowadays.

With a simpler setup there is less chance of bringing everything down with a config change, because things are relatively static after deploy.

In a 'normal' scenario, you might host your web page on an Nginx web server, on a Linux server somewhere. A container lets you do the same thing, but in an isolated area of the operating system. That means you can have an isolated area which has your web pages, the Nginx web server, and some other dependencies, all grouped up together into a distributable package called an image. And then you can pull that image on other servers and just start running it and it'll have your web pages and your Nginx running nicely.

So far that's Docker, or containers to put it more generally.

Now if your web page is so amazing that it receives a lot of traffic, your little container is going to get overwhelmed. And if it falls over, then it's dead and nobody can see your web page until you bring the container back up. Fortunately there are tools that let you manage this aspect of the image, called orchestration. You can tell orchestration tools how to figure out if an image is unhealthy and needs replacement, and if it falls over whether to bring it back, and importantly, how many copies of the image to run to handle the traffic. And if you need to push an updated image with an updated web page in it, how to gently make that new container available to the world without interrupting the traffic.

There's more to orchestration, you also tell containers how to talk to each other if needed, how to manage secrets, encryption, load balancing. There are lots of aspects of hosting that fall into this.

The two main orchestration tools I know of are Docker Swarm and Kubernetes. Docker Swarm is bundled with Docker already. It's pretty easy to shift from normal Docker use to Docker Swarm use, it works well enough for small-medium deployments. Kubernetes is a tool for much larger and highly flexible use cases, and it has a lot of levers and buttons and swiss army knives with its own swiss army knives. Many aspects of Kubernetes like the load balancing and secrets are all pluggable and you can use different tools in there.

Now you're at this article's topic, which is Kubernetes. K8s as it's called has a larger mindshare of the ops world, therefore everyone wants to use it, but it's very complicated, so a tool has been introduced to try to simplify it.

I get what you're saying, but my point is a bit more nuanced:

If your goal is to build highly reliable and available services to end users that are secure and scalable with a team of more than 10 engineers, eventually you will run into more than 50% of the concepts in Kubernetes anyway and end up re-inventing them.

Scaling up and down, node draining, finding out whether services are healthy, RBAC, resource distribution, secrets management, service hardening, introspection capabilities, explicit declaration of dependencies and endpoints and many, many more.

My point is: Sure, if your goal isn't that, it doesn't make sense to start out using Kubernetes.

But if at least eventually that's what you need, imho it's way preferable to just learn and apply well proven abstractions instead of reinventing the wheel along the way and end up with a less maintainable, capable and standardized solution you won't find anyone for maintaining.

If I hear about some of the comments here suggesting to "just spinning up docker-compose with Traefik in front" (disclaimer: I really like Traefik), then that reminds me of how some of the ops mess started that I historically had to care for.