Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Didn't Need Kubernetes, and You Probably Don't Either

benhouston3d.com

Post reply on HN
Print view View original

191–200 of 436 posts

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#191

Post by datadeft »

stiray wrote: post #117
tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I agree, but what pisses me the most is that today higher level abstractions (like cloud, spring boost,...) are hiding lower level functionality so well, that you are literally forced to use obnoxious amounts of time to study documentation (if you are in luck and it is well written), while everything is decorated with new naming of known concepts that was invented by people who didn't know that the concept already ex…

The real benefits of LLMs is to give me a good summary of these documents. Still, using these abstractions is probably not worth it.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#192

Post by DanielHB »

dikei wrote: post #61
paxys wrote: post #48

> Kubernetes comes with substantial infrastructure costs that go beyond DevOps and management time. The high cost arises from needing to provision a bare-bones cluster with redundant management nodes. That's your problem right there. You really don't want to be setting up and managing a cluster from scratch for anything less than a datacenter-scale operation. If you are already on a cloud provider just use their mana…

Even for on-prem scenario, I'd rather maintain a K8S control plane and let developer teams manage their own apps deployment in their own little namespace, than provisioning a bunch of new VMs each time a team need some services deployed.

> than provisioning a bunch of new VMs each time a team need some services deployed.

Back in the old days before cloud providers this was the only option. I started my career in early 2010s and got the tailend of this, it was not fun.

I remember my IT department refusing to set up git for us (we were using SVN before) so we just asked a VM and set up a git repo in there ourselves to host our code.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#193

Post by karmarepellent »

voidfunc wrote: post #107
tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I'm always kind of blown away by experiences like this. Admittedly, I've been using Kubernetes since the early days and I manage an Infra team that operates a couple thousand self-managed Kubernetes clusters so... expert blindness at work. Before that I did everything from golden images to pushing changes via rsync and kicking a script to deploy. Maybe it's because I adopted early and have grown with the technology i…

Agreed. The best thing we did back when we ran k8s clusters, was moving a few stateful services to dedicated VMs and keep the clusters for stateless services (the bulk) only. Running k8s for stateless services was an absolute bliss.

At that time stateful services were somewhat harder to operate on k8s because statefulness (and all that it encapsulates) was kinda full of bugs. That may certainly have changed over the last few years. Maybe we just did it wrong. In any case if you focused on the core parts of k8s that were mature back then, k8s was (and is) a good thing.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#194

Post by jq-r »

voidfunc wrote: post #107
tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I'm always kind of blown away by experiences like this. Admittedly, I've been using Kubernetes since the early days and I manage an Infra team that operates a couple thousand self-managed Kubernetes clusters so... expert blindness at work. Before that I did everything from golden images to pushing changes via rsync and kicking a script to deploy. Maybe it's because I adopted early and have grown with the technology i…

Those "necessary" add-ons and sidecars are out of control, but its the people problem. I'm part of the infra team and we manage just couple of k8s clusters, but those are quite big and have very high traffic load. The k8s + terraform code is simple, with no hacks, reliable and easy to upgrade. Our devs love it, we love it too and all of this makes my job pleasant and as little stressful as possible.

But we recently hired a staff engineer to the team (now the most senior) and the guy just cannot rest still. "Oh we need a service mesh because we need visibility! I've been using it on my previous job and its the best thing ever." Even though we have all the visibility/metrics that we need and never needed more than that. Then its "we need a different ingress controller, X is crap Y surely is much better!" etc.

So its not inexperienced engineers wanting newest hotness because they have no idea how to solve stuff with the tools they have, its sometimes senior engineers trying to justify their salary, "seniority" by buying into complexity as they try to make themselves irreplaceable.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#195

Post by yungporko »

misswaterfairy wrote: post #133
tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I hate "cloud shit" as well, though specifically that there's a vendor-specific 'app', or terminology, or both, for everything that we've had standard terms for, for decades . I just want a damn network, a couple of virtual machines, and a database. Why does each have to create different fancy wrappers over everything, that not even their own sales consultants, and even engineers, understand?(1) What I do like about…

agreed, cloud shit can fuck off. they market themselves as the solution to problems you will never have so that you build your shit on their land and they can charge you rent, and most people eat it up and think they're doing them a favour.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#196

Post by winwang »

semitones wrote: post #15

Kubernetes has a steep learning curve, and certainly a lot of complexity, but when used appropriately for the right case, by god it's glorious

Using it to run easily-scalable Spark clusters. Previously used it for large distributed builds. It's been pretty great (even if annoying at times).

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#197

Post by darkstar_16 »

tayo42 wrote: post #131
voidfunc wrote: post #128

Earlier quoted context omitted.

> So you don't run any databases in those thousands of clusters? We do, but not of the SQL variety (that I am aware of). We have persistent key-value and document store databases hosted in these clusters. SQL databases are off-loaded to managed offering's in the cloud. Admittedly, this does simplify a lot of problems for us.

How much data? I keep hearing k8s isn't usable becasue sometimes there is to much data and it can't be moved around.

In the managed k8s space, the data is on a PVC in the same availability zone as the node it is being mounted on. If the node dies, the volume is just mounted on to a new node in the same zone. There is no data movement required.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#198

Post by vrighter »

cess11 wrote: post #161
tapoxi wrote: post #102

Earlier quoted context omitted.

This read as "old man yells at cloud" to me. I've managed a few thousand VMs in the past, and I'm extremely grateful for it. An image is built in CI, service declares what it needs, the scheduler just handles shit. I'm paged significantly less and things are predictable and consistent unlike the world of VMs where even your best attempt at configuration management would result in drift, because the CM system is only…

I'd prefer Ansible if I was running VM:s. Did that at a gig, controlled a vCenter cluster and hundreds of machines in it, much nicer experience than Kubernetes style ops. Easier to do ad hoc troubleshooting and logging for one.

until, as happened to us, you're in the middle of an upgrade cycle, with a mix of red-hat 6 and red-hat 8 servers, and ansible decide to require support for the latest available version of python on red-hat 8, which isn't available on red-hat 6, so we have no way of using ansible to manage both sets of servers.

The python ecosystem is a cancer.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#199

Post by friendzis »

tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

In my limited experience k8s is treated as some magic instead of what it is: resource virtualization platform. Sort one (or two) step above hardware virtualization platforms (e.g. esxi).

The fact that vmware can migrate a running vm to a different hardware node is surely powerful feature. Do you want to pay for that with complexity? If you are one infra team serving on-prem deployments with consistent loads you provision things when new projects are started and things break. However, if infra team serves internal product teams it is nice to give certain guarantees to them and limit blast radius how they can affect other teams.

This is where kubernetes sit. It's a deployment platform, where the deployable is container image instead of VM image. Slice off a namespace to an internal team and have their deployment blast radius contained within their namespace.

Do you need such flexibility? I'm pretty sure that roughly 99% of all teams do not. A static set of VMs provisioned with some container orchestration system is more than enough for them. Loads and deployments are too static for it to matter.

>>> But it allows seamless upgrades!

Dude or dudette, if you can't properly drain your nodes and migrate sessions now, kubernetes will not save you.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#200

Post by Imustaskforhelp »

Interesting , I was reading the author's comments in the hackernews post about https://www.gitpod.io/blog/we-are-leaving-kubernetes

And I was there also curious about it , and had posted some questions , though now I am not sure if all of those were interested.

But yeah , I also agree with this sentiment , personally I would much rather optimize my code in hopes that it can run on single machine (vps like hetzner) instead of the dread of kubernetes

Though I also want to see kubernetes , so maybe some day I am going to experiment it for the sake of experimentation with something like kubectl or other minimalist single computer based approach to learn new things.

I also feel like I can't benchmark things unless done through kubernetes like anton putra , so I am not sure.

Post reply on HN