Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Didn't Need Kubernetes, and You Probably Don't Either

benhouston3d.com

Post reply on HN
Print view View original

201–210 of 436 posts

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#201

Post by torginus »

tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

Kubernetes starts sucking at first sight and is kind of the sign of the ailment of modern times.

Let me try to explain:

First, you encounter the biggest impedance mismatch between cloud and on prem: Kubernetes works with pods, while AWS works with instances as the unit of useful work, so they must map to each other, right?

Wrong, first each instance needs to run a Kubernetes node, which duplicates the management infrastructure hosted by AWS, and reduces the support for granularity, like if I need 4 cores for my workload, I start an instance with 4 cores, right?. Not so with k8s, you have to start up a big node, then schedule pods there.

Yay, extra complexity and overhead! And it's like when you need 3 carrots for the meal your cooking, but the store only sells it in packs of 8, you have to pay for the ones you don't need, and then figure out how to utilize the waste.

I'm not even going to talk about on-prem kubernetes, as I've never seen anyone IRL use it.

Another thing is that the need for kubernetes is manifestation of crappy modern dev culture. If you wrote your server in node, Python and Ruby, you're running a single threaded app in the era of even laptops having 32 core CPUs. And individual instances are slow, so you're even more dependent on scaling.

So, to make use of the extra CPU power, you're forced to turn to Docker/k8s and scale your infra that way, whereas if you went with something like Go, or god forbid, something as deeply uncool as ASP.NET, you could just put your 32 core server, and you get fast single threaded perf, and perfect and simple multi-threaded utilization out of the box without any of the headaches.

Also I've found stuff like rolling updates to be a gimmick.

Also a huge disclaimer, I don't think k8s is a fundamentally sucky or stupid thing, it's just I've never seen it used as a beneficial architectural pattern in practice.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#202

Post by cess11 »

vrighter wrote: post #198
cess11 wrote: post #161

Earlier quoted context omitted.

I'd prefer Ansible if I was running VM:s. Did that at a gig, controlled a vCenter cluster and hundreds of machines in it, much nicer experience than Kubernetes style ops. Easier to do ad hoc troubleshooting and logging for one.

until, as happened to us, you're in the middle of an upgrade cycle, with a mix of red-hat 6 and red-hat 8 servers, and ansible decide to require support for the latest available version of python on red-hat 8, which isn't available on red-hat 6, so we have no way of using ansible to manage both sets of servers. The python ecosystem is a cancer.

Sure, and I'm also not a fan of RedHat. We ran Ubuntu and Debian on that gig, the few Python issues we ran into we could fix with some package pinnings and stuff like that.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#204

Post by eek04_ »

tayo42 wrote: post #131
voidfunc wrote: post #128

Earlier quoted context omitted.

> So you don't run any databases in those thousands of clusters? We do, but not of the SQL variety (that I am aware of). We have persistent key-value and document store databases hosted in these clusters. SQL databases are off-loaded to managed offering's in the cloud. Admittedly, this does simplify a lot of problems for us.

How much data? I keep hearing k8s isn't usable becasue sometimes there is to much data and it can't be moved around.

While I've not played with k8, I did run stuff in Google's Borg for a very long while, and that has a similar architecture. My team was petabyte scale and we were far from the team with the largest footprint. So it is clearly possible to handle large scale data in this type of architecture.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#205

Post by davidgl »

ants_everywhere wrote: post #42

People talk about Kubernetes as container orchestration, but I think that's kind of backwards. Kubernetes is a tool for creating computer clusters. Hence the name "Borg" (Kubernetes's grandpa) referring to assimilating heterogeneous hardware into a collective entity. Containers are an implementation detail. Do you need a computer cluster? If so k8s is pretty great. If you don't care about redundancy and can get all t…

Yep, it's a cluster OS. If you need to run a cluster, you need to explore and understand the trade offs of k8s versus other approaches. Personally I run a small cluster on k3s, for internal tools, and love it. Yes it's a load of new abstractions to learn, but once learnt if really helps in designing large scalable systems. I manage lots of pet machines and VMs for clients, and it would be soooo much easier on k8.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#206

Post by jmb99 »

tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I have pretty much the exact same opinion, right down to “cloud shit” (I have even used that exact term at work to multiple levels of management, specifically “I’m not working on cloud shit, you know what I like working on and I’ll do pretty much anything else, but no cloud shit”). I have worked on too many projects where all you need is a 4-8 vCPU VM running nginx and some external database, but for some reasons there’s like 30 containers and 45 different repos and 10k lines of terraform, and the fastest requests take 750ms which should be nginx->SQL->return in But for some reason this is what people want to do. They would rather spend hours debugging kubernetes, terraform, and docker, and spending 5 digits on cloud every month, to serve what could literally be proxied authenticated DB lookups. We have “hack days” a few times a year, and I’m genuinely debating rewriting the entire “cloud” portion of our current product in gunicorn or something, host it on a $50/month vps, point it at a mirror of our prod db, and see how many orders of magnitude of performance I can knock off in a day.

I’ve only managed to convert one “cloud person” to my viewpoint but it was quite entertaining. I was demoing a side project[0] that involved pulling data from ~6 different sources (none hosted by me), concatenating them, deduping them, doing some math, looking up in a different source an image (unique to each displayed item), and then displaying the list of final items with images in a list or a grid. ~5k items. Load time on my fibre connection was 200-250ms, sorting/filtering was I think it’s so heavily ingrained in most cloud/web developers that this is the only option that they will not even entertain the thought that it can be done another way.

[0] This particular project failed for other reasons, and is no longer live.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#207

Post by psini »

Spivak wrote: post #65
ants_everywhere wrote: post #42

People talk about Kubernetes as container orchestration, but I think that's kind of backwards. Kubernetes is a tool for creating computer clusters. Hence the name "Borg" (Kubernetes's grandpa) referring to assimilating heterogeneous hardware into a collective entity. Containers are an implementation detail. Do you need a computer cluster? If so k8s is pretty great. If you don't care about redundancy and can get all t…

This has got to be the most out there k8s take I've read in a while. k8s doesn't save you from learning your cloud providers infrastructure, you have to learn k8s in addition to your cloud provider's infrastructure. It's all ALBs, ASGs, Security Groups, EBS Volumbes and IAM policy underneath and k8s, while very clever, isn't so clever as to abstract much of any of it away from you. On EKS you get to enjoy more odd li…

You can self host Kubernetes on "dumb" VMs from Hetzner or OVH.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#209

Post by mrweasel »

tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I wouldn't throw "cloud shit" in the same bucket as Kubernetes. Professional cloud services are mostly great, but super expensive.

Kubernetes is interesting, because it basically takes everything you know and sort of pushes it down the stack (or up, depending on your viewpoint). To some extend I get the impression that the idea was: Wouldn't it be great if we took all the infrastructure stuff, and just merged the whole thing into one tool, which you can configure using a completely unsuitable markup language. The answer is that "No, that would in fact not be great".

For me the issue is that Kubernetes is overused. You can not really argue that it's not useful or has its place, but that place is much much small than the Internet wants us to believe. It's one of the few services where I feel like "Call us" would be an appropriate sales method.

The article is correct, you probably don't need Kubernetes. It's a amazing piece of technology, but it's not for the majority of us. It is and should be viewed as a niche product.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#210

Post by carlmr »

voidfunc wrote: post #107
tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I'm always kind of blown away by experiences like this. Admittedly, I've been using Kubernetes since the early days and I manage an Infra team that operates a couple thousand self-managed Kubernetes clusters so... expert blindness at work. Before that I did everything from golden images to pushing changes via rsync and kicking a script to deploy. Maybe it's because I adopted early and have grown with the technology i…

>expert blindness at work.

>It's not that complicated if you limit yourself to the core stuff.

Isn't this the core problem with a lot of technologies. There's a right way to use it, but most ways are wrong. An expert will not look left and right anymore, but to anyone entering the technology with fresh eyes it's a field with abundance of landmines to navigate around.

It's simply bad UX and documentation. It could probably be better. But now it's too late to change everything because you'd annoy all the experts.

>There's way too much vendor garbage surrounding the ecosystem

Azure has been especially bad in this regard. Poorly documented in all respects, too many confusing UI menus that have similar or same names and do different things. If you use Azure Kubernetes the wrapper makes it much harder to learn the "core essentials". It's better to run minkube and get to know k8s first. Even then a lot of the Azure stuff remains confusing.

Post reply on HN