Live data from Hacker News

Google admits Kubernetes container tech is too complex

theregister.com

191–200 of 449 posts

Re: Google admits Kubernetes container tech is too complex

#191

I remember trying out docker sometime back in late 2013. Something about it never fully stuck with me. I always felt like the final boundary for a piece software should be the process, not the computer. Plopping an entire VM into a zipfile and saying "here's your software" felt like lazy engineering to many of us at the time (and still today). For our current stack, the answer has been to make the entire business app…

After that single node crashes, the app you're running on that one server will run a little more slowly..

How big should the one server that serves the whole of netflix be..?

Re: Google admits Kubernetes container tech is too complex

#192
post #164

Earlier quoted context omitted.

Thanks, that's really interesting. Everyone has different challenges and requirements, and of course different experiences. For smaller setups (say 1-10 services) I'm quite happy with cloud config and one VM per process behind one load balancer per service. It's simple to set up, scale and reproduce. This setup doesn't autoscale, but I've never really felt the need. We use Go and deploy one static binary per service…

>We use Go and deploy one static binary per service at work with minimal dependencies so docker has never been very interesting. how do you deploy your static binary to the server? (without much downtime ?)

> how do you deploy your static binary to the server? (without much downtime ?)

Ask yourself how would you solve this problem if you deployed by hand and automate that.

1. Create a brain-dead registry that gets information about what runs where (service name, ip address:port number, id, git commit, service state, last healthy_at). If you want to go crazy, do it 3x.

2. Have haproxy or nginx use the registry to build a communication map between services.

You are done.

For extra credit ( which is nearly cost free ) with 1. you now can build a brain-dead simple control plane by sticking an interface to 1 that lets someone/something toggle services automatically. For example, if you add percentage gauge to services, you can do hitless rolling deploys or cannery deploys.

Re: Google admits Kubernetes container tech is too complex

#193

I remember trying out docker sometime back in late 2013. Something about it never fully stuck with me. I always felt like the final boundary for a piece software should be the process, not the computer. Plopping an entire VM into a zipfile and saying "here's your software" felt like lazy engineering to many of us at the time (and still today). For our current stack, the answer has been to make the entire business app…

After that single node crashes, the app you're running on that one server will run a little more slowly.. How big should the one server that serves the whole of netflix be..?

"You can handle millions of simultaneous clients without blinking. Who even has that many total customers right now?"

Apparently netflix has that many customers. Then again, if you split Netflix into regions and separate all the account logic from the streaming, the recommendations engine and the movie-content, you could perhaps run the account logic for one region in one server.

Re: Google admits Kubernetes container tech is too complex

#194

I remember trying out docker sometime back in late 2013. Something about it never fully stuck with me. I always felt like the final boundary for a piece software should be the process, not the computer. Plopping an entire VM into a zipfile and saying "here's your software" felt like lazy engineering to many of us at the time (and still today). For our current stack, the answer has been to make the entire business app…

> Horizonal scalability is simply a band-aid for poor engineering in most (not all) applications.

Maybe if your app handles < 10k concurrent connections. Otherwise it is the most cost efficient solution and exists because it solves the scaling problem in the best way as of today.

Re: Google admits Kubernetes container tech is too complex

#195

Putting on my Asbestos Longjohns: The more I look into k8s ecosystem, the more I'm convinced that it's one of those things that suits FAANG etc, but the regular Joe developer has caught on the fad and wants to add it to his repertoire, even though it's an overkill. After all no one got fired for buying IBM and recommending Kubernetes. Most teams need a simpler deployment strategies that other's have succinctly mentio…

I think Kubernetes in principle gets a lot of things very right - but it has over time grown into this huge amorphous blob of complexity that makes it very easy to shoot yourself in the foot with, as many people said :)

That issue is not endemic to Kubernetes, but rather to any larger system past a certain age, you learn stuff as you go along and would do stuff differently if you did it again today - but you can't easily, because you cannot break compatibility for everybody using your stuff.

As a concrete example from the Kubernetes world, there is a talk by Tim Hockin [1] about how today, they would fundamentally design the api-server differently and base pretty much everything on CRDs.

[1] https://www.youtube.com/watch?v=ji0FWzFwNhA

Re: Google admits Kubernetes container tech is too complex

#196

I remember trying out docker sometime back in late 2013. Something about it never fully stuck with me. I always felt like the final boundary for a piece software should be the process, not the computer. Plopping an entire VM into a zipfile and saying "here's your software" felt like lazy engineering to many of us at the time (and still today). For our current stack, the answer has been to make the entire business app…

Containers solve the dependency problem by simply pretending it doesn't exist.

I used to do UNIX integration work in the late 1990's early 2000's and containers weren't really a thing. So you had to make sure libs from one program didn't crap on another program. And developers had to be conscious of what dependencies they included in their code. Nowadays they don't have to care as much because of containers. Every program can have its own dependencies. Thereby solving the integration problem.

A better solution would be to actually integrate programs and their dependencies into working systems, but no one has time for that. Software bloat is fine. Computers are cheap and fast. And actually understanding what we're doing would be too expensive. So just wrap all your have finished crapware up in a giant black box and dump it on a server.

Re: Google admits Kubernetes container tech is too complex

#198
K8s is amazing piece of tech, but still there is a big cost and complexity inherent in adopting & managing it.

I enjoy using it and playing with it, but so many use cases can be addressed with something simpler - either just Docker / Swarm / AWS ECS etc. alternatives or just going for VMs with well defined CI/CD processes that let you tear down the infrastructure and set it up again easily.

What very interests me are the concepts K8s build on that are not usually recognized - to me K8s seems a lot like a JVM, just that it operates on infrastructural (and not runtime) level.

I enjoy experimenting with these concepts when applied back in the runtime world - it is for instance interesting to run 100s of servers with JVM and let them load/execute new dependencies and code at runtime (JVM is very well suited for that).

This is area that is not yet explored and would probably deserve more attention as it allows for distributed rapid computing that is infrastructure/platform independent (the downside is that it requires (just) JVM and the isolation is not perfect).

Re: Google admits Kubernetes container tech is too complex

#199
Looks like a bunch of people on this thread missed the point of containers being more lightweight than VMs, more compute efficient than VMs and physical machines.

Dedicated servers are heavily under utilized and over provisioned because once you allot a server to a team they don't want to give it back.

VM solved this problem and changed how servers were provisioned. Docker and K8s are the next progression of this. People who compare K8s to the 'next js framework' have to do some serious context alignment...

Re: Google admits Kubernetes container tech is too complex

#200

Putting on my Asbestos Longjohns: The more I look into k8s ecosystem, the more I'm convinced that it's one of those things that suits FAANG etc, but the regular Joe developer has caught on the fad and wants to add it to his repertoire, even though it's an overkill. After all no one got fired for buying IBM and recommending Kubernetes. Most teams need a simpler deployment strategies that other's have succinctly mentio…

From my experience it's actually sold as a simpler alternative to other infra provisioning. So you end up with situations where a team deploys whatever with a helm chart, and it sets up the stuff like magic and they build on it. Then when something goes wrong they literally have no idea how to fix anything and it becomes a waking nightmare.
Post reply on HN