Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Didn't Need Kubernetes, and You Probably Don't Either

benhouston3d.com

Post reply on HN
Print view View original

231–240 of 436 posts

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#231

Post by throwaway2037 »

jmb99 wrote: post #206
tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I have pretty much the exact same opinion, right down to “cloud shit” (I have even used that exact term at work to multiple levels of management, specifically “I’m not working on cloud shit, you know what I like working on and I’ll do pretty much anything else, but no cloud shit”). I have worked on too many projects where all you need is a 4-8 vCPU VM running nginx and some external database, but for some reasons the…

These comments:

    > He didn’t believe it was possible to have this kind of performance in a “legacy monolithic” (his words) application.

    > I think it’s so heavily ingrained in most cloud/web developers that this is the only option that they will not even entertain the thought that it can be done another way.
One thing that I need to remind myself of periodically: The amount of work that a modern 1U server can do in 2024 is astonishing.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#232

Post by erinaceousjones »

torginus wrote: post #201
tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

Kubernetes starts sucking at first sight and is kind of the sign of the ailment of modern times. Let me try to explain: First, you encounter the biggest impedance mismatch between cloud and on prem: Kubernetes works with pods, while AWS works with instances as the unit of useful work, so they must map to each other, right? Wrong, first each instance needs to run a Kubernetes node, which duplicates the management infr…

Being nitpicky about Python specifically, Python not necessarily single-threaded; gunicorn gets you a multiprocess HTTP/WSGI server if you configure it to. asyncio and gevent have made it easier to do things actually-concurrently. Judicious use of generator functions lets you stream results back instead of blocking I/O for big chunks. And we have threads. Yeah, the Global Interpreter Lock is still hanging around, it's not the fastest language, but there are ways to produce a Docker image of an API written in Python which can handle thousands of concurrent HTTP requests and actively use all available CPU cores to do intensive computation.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#233

Post by quietbritishjim »

voidfunc wrote: post #107
tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I'm always kind of blown away by experiences like this. Admittedly, I've been using Kubernetes since the early days and I manage an Infra team that operates a couple thousand self-managed Kubernetes clusters so... expert blindness at work. Before that I did everything from golden images to pushing changes via rsync and kicking a script to deploy. Maybe it's because I adopted early and have grown with the technology i…

I hardly know the first thing about Kubernetes or cloud, so maybe you can help explain something to me:

There's another Kubernetes post on the front page of HN at the moment, where they complain it's too complex and they had to stop using it. The comments are really laying into the article author because they used almost 50 clusters. Of course they were having trouble, the comments say, if you introduce that much complexity. They should only need one single cluster (maybe also a backup and a dev one at most). That's the whole point.

But then here you are saying your team "operates a couple thousand" clusters. If 50 is far too many, and bound to be unmanageable, how is it reasonable to have more than a thousand?

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#234

Post by dikei »

sbstp wrote: post #136
paxys wrote: post #48

> Kubernetes comes with substantial infrastructure costs that go beyond DevOps and management time. The high cost arises from needing to provision a bare-bones cluster with redundant management nodes. That's your problem right there. You really don't want to be setting up and managing a cluster from scratch for anything less than a datacenter-scale operation. If you are already on a cloud provider just use their mana…

Most control planes are not free anymore, they cost like 70$/mo on AWS & GCP. Used to be a while back.

GCP has $74 free credit for Zonal cluster, so you effectively have the first cluster for free.

And even $70 is cheap, considering that a cluster should be shared by all the services from all the teams in the same environment, bar very few exceptions.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#235

Post by PittleyDunkin »

bluehatbrit wrote: post #185
ozim wrote: post #154

Earlier quoted context omitted.

Mostly business guys don’t know the difference but we are running on local cloud provider and they think of it is not on Azure or AWS it is not in cloud - they understand that we run stuff on servers but they also don’t understand VPS is IaaS. Developers want to use PaaS and also AWS or Azure so they can put it on their resume for the future.

> Developers want to use PaaS and also AWS or Azure so they can put it on their resume for the future. I think this is a little disingenuous. Developers want to use them because they already know them. The services composing them are also often well documented by the provider. I say all of that as someone trying to move a company away from aws, and over to our own hardware.

I want to use them because I'm not paying for it and managing your own hardware is a pain in the ass I'd rather avoid when I could just code instead. Which is not to say that using your own hardware isn't smart, but it is definitely miserable.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#236

Post by vergessenmir »

it's baffling to me that realising that your application deployment requirements don't fit k8s means that k8s is not fit for purpose. Note that the alternative isn't a return to the management of VMs, which we all agree none of us ever want to return to, but is a cloud service which hides the concerns that k8s makes explicit.

It's a bit short-sighted (granted not intentionally) to talk about k8s not being fit for purpose without specifying the workloads it's not suitable for and at what scale of operations. Often times when these discussions come up it's usually reduced to a stateless web app being the justification for illustrating the shortfalls of k8s as a solution.

Personally, I'm not a big fan of k8s because it can be complex but the alternatives tend to be more complex. I can guarantee, your solution over time will converge on operational knowledge trapped in the minds of your team members , runbooks and custom scripts that express an abstraction that k8s implements perfectly well.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#238

Post by everfrustrated »

secondcoming wrote: post #218
kortilla wrote: post #212

Earlier quoted context omitted.

All of these anecdotes seem to come from people who don’t bother to try to learn kubernetes. > YAML files, and then spend a day fixing them by copy-pasting increasingly-convoluted things on stackexchange. This is terrible behavior. Its not any different from yanking out pam modules because you’re getting SSH auth failures caused by a bad permission on an SSH key. > If I get to tens of millions of users, maybe I’ll wo…

Just because you're using VMs doesn't mean you're now dealing with state. It's 100% possible to have stateless VMs running in an auto-scaling instance group (in GCP speak, I forget what AWS calls them)

In the beginning AWS didn't even support state on their VMs! All VMs were ephemeral with no state persistence when terminated. They later introduced EBS to allow for the more classic enterprise IT use cases.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#239

Post by MortyWaves »

devjab wrote: post #121
tombert wrote: post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I don’t mind the cloud, but even in enterprise organisations I fail to see the value of a lot of the more complex tools. I’ve anlways worked with Azure because Denmark is basically Microsoft territory in a lot of non-tech organisations because of the synergy between pricing and IT operations staff. I’ve done bicep, terraform and both Kubernetes and the managed (I forgot what azure conteiner apps running on top of wha…

This was a good read! I have similar thoughts especially about IaC vs a bash script. Definitely clear pros and cons to both, but I’m wondering how you handle infrastructure drift with imperative bash scripts?

I mean hopefully no one is logging into Azure to fuck with settings but I’m sure we’ve all worked with that one team that doesn’t give a flying fuck about good practices.

Or say you wish to now scale up a VM, how does your bash script deal with that?

Do you copy past the old script, pass new flags to the Azure CLI, and then run that, then delete the old infrastructure somehow?

I’m curious because I think I’d like to try your approach.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#240

Post by pas »

AtlasBarfed wrote: post #125
voidfunc wrote: post #107

Earlier quoted context omitted.

I'm always kind of blown away by experiences like this. Admittedly, I've been using Kubernetes since the early days and I manage an Infra team that operates a couple thousand self-managed Kubernetes clusters so... expert blindness at work. Before that I did everything from golden images to pushing changes via rsync and kicking a script to deploy. Maybe it's because I adopted early and have grown with the technology i…

So you don't run any databases in those thousands of clusters? To your point, and I have not used k8s I just started to research it when my former company was thinking about shoehorning cassandra into k8s... But there was dogma around not allowing access to VM command exec via kubectl, while I basically needed it in the basic mode for certain one-off diagnosis needs and nodetool stuff... And yes, some of the floated…

postgresql operators are pretty nice, so it makes sense to run stateful stuff on k8s (ie. for CI, testing, staging, dev, etc.. and probably even for prod if there's a need to orchestrate shards)

> exec

kubectl exec is good, and it's possible to audit access (ie. get kubectl exec events with arguments logged)

and I guess and admissions webhook can filter the allowed commands

but IMHO it's shouldn't be necessary, the bastion host where the "kubectl exec" is run from should be accessible only through an SSH session recorder

Post reply on HN