Live data from Hacker News

I Didn't Need Kubernetes, and You Probably Don't Either

benhouston3d.com

121–130 of 436 posts

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#121
post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I don’t mind the cloud, but even in enterprise organisations I fail to see the value of a lot of the more complex tools. I’ve anlways worked with Azure because Denmark is basically Microsoft territory in a lot of non-tech organisations because of the synergy between pricing and IT operations staff.

I’ve done bicep, terraform and both Kubernetes and the managed (I forgot what azure conteiner apps running on top of what is basically Kubernetes is called). When I can get away with it I always use the Azure CLI through bash scripts in a pipeline however and build directly into Azure App services for contained which is just so much less complicated than what you probably call “cloud shit”. The cool part about the Azure CLI and their app services is that it hasn’t really changed in the past 3 years, and they are almost one size fit any organisation. So all anyone needs to update in the YAML scripts are the variables. By contrast working with Bicep/Terraform, Jenkins and whatever else people use has been absolutely horrible, sometimes requiring full time staff just to keep it updated. I suppose it may be better now that azure co-pilot can probably auto-generate what you need. A complete waste of resources in my opinion. It used to be more expensive, but with the last price hike of 600% on azure container apps it’s usually cheaper. It’s also way more cost efficient in terms of maintaining since it’ll just work after the initial setup pipeline has run. This is the only way I have found that is easier than what it was when organisations ran their own servers. Whether it was in the basement or at some local hardware house (not exactly sure what you call the places where you rent server rack space). Well places like Digital Ocean are even easier but they aren’t used by enterprise.

I’m fairly certain I’ve ever worked with an organisation that needed anything more than that since basically nothing in Denmark scales beyond what can run on a couple of servers behind a load balancer. One of the few exceptions is the tax system which sees almost 0 usage except for the couple of weeks where the entire adult population logs in in at the same time. When DevOps teams push back, I tend to remind them that StackOverflow ran on a couple of IIS servers for a while and that they don’t have even 10% of the users.

Eventually the business case for Azure will push people back to renting hardware space or jumping to Hetzner and similar. But that’s a different story.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#122
RE: slow autoscaling

Maybe the cloud companies could do something here by always keeping a small subset of machines online and ready to join the cluster. Provided there is some compromise in what the configuration is for the end user. I guess it doesn't solve image pulling. Pre-warming nodes is an annoying problem to solve.

Best solution I've been able to come up with is: Spegel (lightweight p2p image caching) + Karpenter (dynamic node autoscaling) + pods with low priority to hold onto some extra nodes. It's not perfect though

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#123

People talk about Kubernetes as container orchestration, but I think that's kind of backwards. Kubernetes is a tool for creating computer clusters. Hence the name "Borg" (Kubernetes's grandpa) referring to assimilating heterogeneous hardware into a collective entity. Containers are an implementation detail. Do you need a computer cluster? If so k8s is pretty great. If you don't care about redundancy and can get all t…

Exactly. I've come to describe k8s as a distributed operating system for servers.

K8s tries to abstract away individual "servers" and gives you an API to interact with all the compute/storage in the cluster.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#125
post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I'm always kind of blown away by experiences like this. Admittedly, I've been using Kubernetes since the early days and I manage an Infra team that operates a couple thousand self-managed Kubernetes clusters so... expert blindness at work. Before that I did everything from golden images to pushing changes via rsync and kicking a script to deploy. Maybe it's because I adopted early and have grown with the technology i…

So you don't run any databases in those thousands of clusters?

To your point, and I have not used k8s I just started to research it when my former company was thinking about shoehorning cassandra into k8s...

But there was dogma around not allowing access to VM command exec via kubectl, while I basically needed it in the basic mode for certain one-off diagnosis needs and nodetool stuff...

And yes, some of the floated stuff was "use sidecars" which also seemed to architect complexity for dogma's sake.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#126

Earlier quoted context omitted.

I'm always kind of blown away by experiences like this. Admittedly, I've been using Kubernetes since the early days and I manage an Infra team that operates a couple thousand self-managed Kubernetes clusters so... expert blindness at work. Before that I did everything from golden images to pushing changes via rsync and kicking a script to deploy. Maybe it's because I adopted early and have grown with the technology i…

This, all the sidecars. Use kubernetes to run your app like you would without it, take advantage of the flexibility, avoid the extra complexity. Service discovery sidecars? Why not just use the out of the box dns features?

Because new people don't know better - I've never used k8s, but have seen sidecars being promoted as a good thing, so I might have used them

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#127
post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I'm always kind of blown away by experiences like this. Admittedly, I've been using Kubernetes since the early days and I manage an Infra team that operates a couple thousand self-managed Kubernetes clusters so... expert blindness at work. Before that I did everything from golden images to pushing changes via rsync and kicking a script to deploy. Maybe it's because I adopted early and have grown with the technology i…

I would pay for the outline of this book.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#128

Earlier quoted context omitted.

I'm always kind of blown away by experiences like this. Admittedly, I've been using Kubernetes since the early days and I manage an Infra team that operates a couple thousand self-managed Kubernetes clusters so... expert blindness at work. Before that I did everything from golden images to pushing changes via rsync and kicking a script to deploy. Maybe it's because I adopted early and have grown with the technology i…

So you don't run any databases in those thousands of clusters? To your point, and I have not used k8s I just started to research it when my former company was thinking about shoehorning cassandra into k8s... But there was dogma around not allowing access to VM command exec via kubectl, while I basically needed it in the basic mode for certain one-off diagnosis needs and nodetool stuff... And yes, some of the floated…

> So you don't run any databases in those thousands of clusters?

We do, but not of the SQL variety (that I am aware of). We have persistent key-value and document store databases hosted in these clusters. SQL databases are off-loaded to managed offering's in the cloud. Admittedly, this does simplify a lot of problems for us.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#129
One can avoid container orchestration by avoiding the trend of containerizing your app. It wastes system resources, provides half-baked replicas of OS services, reduces overall security while simultaneously making networking a total pitas.

Your cloud provider is already divvying up a racked server into your VPS's, via a hypervisor, then you install an OS on your pretend computer.

While i can see how containerized apps provide a streamlined devops solution for rare hard to configure software that needs to run on Acorn OS 0.2.3 only, it should never be the deployment solution for a public facing production web service.

Horses for courses.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#130
post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I am running VPSes at our small startup-ish company on IaaS cloud.

Every time we get a new guy I have to explain that we are already „in cloud” there is no need to „move to cloud”.

Post reply on HN