Live data from Hacker News

I Didn't Need Kubernetes, and You Probably Don't Either

benhouston3d.com

171–180 of 436 posts

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#171

Earlier quoted context omitted.

> Can you reduce it further? What if you don't require 2 cloud providers? What if you don't need zero-downtime? I think the "2 cloud providers" criteria is maybe negotiable. Also, maybe there was a misunderstanding: I didn't mean to say I want to run it on two cloud providers. But rather that I run it on one of them but I could easily migrate to the other one if necessary. The zero-downtime one isn't. It's not necess…

I disagree with that last part. By default, having a few seconds downtime is not complex. The easiest thing you could do to a server is restart it. Its literally just a restart!

I managed a few production services like this and it added a lot of overhead to my work. On the one hand I'd get developers asking me why their stuff hasn't been deployed yet. But then I'd also have to think carefully about when to deploy and actually watch it to make sure it came back up again. I would often miss deployment windows because I was doing something else (my real job).

I'm sure there are many solutions but K8s gives us both fully declarative infrastructure configs and zero downtime deployment out of the box (well, assuming you set appropriate readiness probes etc)

So now I (a developer) don't have to worry about server restarts or anything for normal day to day work. We don't have a dedicated DevOps/platforms/SRE team or whatnot. Now if something needs attention, whatever it is, I put my k8s hat on and look at it. Previously it was like "hmm... how does this service deployment work again..?"

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#172
I don't get these recent anti-Kubernetes posts, yes if you're deploying a simple app then there are alternatives which are easier, but as your app starts to get more complex then suddenly you'll be wishing you had the Kubernetes API.

I'd use Kubernetes even if I was spinning up a single VM and installing k3s on it. It's a universal deployment target.

Spinning up a cluster isn't the easiest thing, but I don't understand how a lot of the complaints around this come from sysadmin-type people who replace Kubernetes with spinning up VMs instead. The main complexity I've found from managing a cluster is the boring sysadmin stuff, PKI, firewall rules, scripts to automate. Kubernetes itself is pretty rock solid. A lot of cluster failure modes still result in your app still working. Etcd can be a pain but if you want you can even replace that with a SQL database for moderate sized clusters if that's more your thing (easier to manage HA control plane then too if you've already got a HA SQL server).

Or yes just use a managed Kubernetes cluster.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#173
post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

> If I get to tens of millions of users, maybe I’ll worry about it then.

Nope, then you'll set up sharded databases and a bunch of application servers behind a load balancer.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#174
post #162
post #158

Earlier quoted context omitted.

Is nomad still around?

Thanks, hadn't heard of that. Seems pretty active per its commit activity: https://github.com/hashicorp/nomad/graphs/commit-activity But the fact that I hadn't heard of it before makes it sound not very popular, at least not for the bubble I live in :). Does anyone have any practical experiences to share about it?

Yes have a few Nomad clusters in production and it's been great.

You'll certainly want to combine it with Consul and use Consul templates and service discovery though.

I'd say the difficulty and complexity level is between Kubernetes and Docker Swarm, not having to use YML too is a big benefit imho.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#175
The hell of k8s for le are the libs in version 0.12536 and all the frequent updates and compatibility versions you need to maintain between them. Why would anyone want to do that in prod. It feels like at some point the tech needs to mature in something more stable and its not coming

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#176
post #90

I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…

I'm sure you can get some of the handy extras that come with a typical kubernetes deployment without the kubernetes, but overall I'll take kubernetes + cloud. Once you've got the hang of it, it's ok. I have a terraform project that deploys clusters with external-dns, external-secrets, cert-manager, metrics, monitoring stack, scalers and FluxCD. From there, pretty much everything else is done via FluxCD (workloads, dashboards, alerts). And while I detest writing helm charts (and sometimes using them, as they can get "stuck" in several ways) they do allow you to wrap up a lot of the kubernetes components into a single package that accepts more-or-less standardized yaml for stuff like resource limits, annotations (eg for granting and AWS role to a service) etc. And FluxCD .postBuild is extremely handy for defining environment vars to apply to more generic template yaml, so we avoid a sprawl. So much so that I am the one-man-band (Sys|Dev|Sec)Ops for our small company, and that doesn't give me panic attacks.

The cloud integration part can be hairy but I have terraform patterns that, once worked out, are cookie cutter.

With cloud kubernetes, I can imagine starting from scratch, taking a wrong turn and ending up in hell.

But I'm exchanging one problem set for another. Having spent years managing fleets of physical and virtual servers, I'm happier and more productive now. I never need to worry about building systems or automation for doing OS build / patching, config management, application packaging and deployment, secrets management, service discovery, external DNS, load balancering, TSL certs etc. Because while those are just "words" now, back then each one was a huge project involving multiple people fighting over "CentOS Vs Ubuntu", "Puppet Vs Ansible", "RPMs Vs docker containers", "Patching Vs swapping AMIs". If you're using Consul and Vault, good luck - you have to integrate all of that into whatever mess you've built, and you'll likely have to write puppet code and scripts to hook it all up together. I lost a chunk of my life writing 'dockerctl' and a bunch of puppet code that deployed it so it could manage docker containers as systemd services. Then building a vault integration for that. It worked great across multiple data centers, but took considerable effort. And in the end it's a unique snowflake used by exactly one company, hardly documented and likely full of undiscovered bugs and race conditions even after all the hard work. The time it took to onboard new engineers was considerable and it took time away from an existing engineer. And we still had certificates expire in production.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#177
post #7

Earlier quoted context omitted.

It's just the hype moving on. Every generation has to make similar mistakes again and again. I am sure if we had the opportunity and the hype was there we would've used k8s in 2005 as well. The same thing is true for e.g. JavaScript on the frontend. I am currently migrating a project from React to HTMX. Suddenly there is no build step anymore. Some people were like: "That's possible?" Yes, yes it is and it turns out…

> Just as a carpenter doesn't use a hammer to saw, we as a profession should strive to use the right tool for the right job I think this is a gross misunderstanding of the complexity of tools available to carpenters. Use a saw. Sure, electric, hand powered? Bandsaw, chop saw, jigsaw, scrollsaw? What about using CAD to control the saw? > Suddenly there is no build step anymore How do you handle making sure the JS you…

As far as browser prefixes go, you know that browser vendors have largely stopped using those? Not even recently, that process started already way back in 2016. Chances are that if you are using prefixes in 2024 you are supporting browsers versions who, by all logic, should no longer have internet access because of all the security implications....

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#178

Earlier quoted context omitted.

So, let's say you want to deploy server instances. Let's keep it simple and say you want to have 2 instances running. You want to have zero-downtime-deployment. And you want to have these 2 instances be able to access configuration (that contains secrets). You want load balancing, with the option to integrate an external load balancer. And, last, you want to be able to run this setup both locally and also on at least…

I think you are proving the point; there are very, very few applications that need to run on two cloud providers. If you do, sure, use Kubernetes if that makes your job easier. For the other 99% of applications, it’s overkill. Apart from that requirement, all of this is very doable with EC2 instances behind an ALB, each running nginx as a reverse proxy to an application server with hot restarting (e.g. Puma) launched…

Hmm, let's see, so you've got to know: EC2, ALB, Nginx, Puma, Systemd, then presumably something like Terraform and Ansible to deploy those configs, or write a custom set of bash scripts. And all of that and you're tied to one cloud provider.

Or, instead of reinventing the same wheels for Nth time, I could just use a set of abstractions that work for 99% of network services out there, on any cloud or bare metal. That set of abstractions is k8s.

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#179
post #120

Earlier quoted context omitted.

I'm always kind of blown away by experiences like this. Admittedly, I've been using Kubernetes since the early days and I manage an Infra team that operates a couple thousand self-managed Kubernetes clusters so... expert blindness at work. Before that I did everything from golden images to pushing changes via rsync and kicking a script to deploy. Maybe it's because I adopted early and have grown with the technology i…

I would buy the book. Just translate all "new language" concepts into well known concepts from networking and system administration. It would be best seller. If I would only have a penny for each time I wasted hours trying to figure out what something in "modern IT" is, just to figure out that I already knew what it is, but it was well hidden under layers of newspeak...

This[0] is my take on something like that, but I'm no k8s expert -- the post documents my first contact with k8s and what/how I understood these concepts, from a sysadmin/SWE perspective.

[0]: https://blog.davidv.dev/posts/first-contact-with-k8s/

Re: I Didn't Need Kubernetes, and You Probably Don't Either

#180

Earlier quoted context omitted.

Sorry to have to tell you this, but you’re old. Your neural plasticity has gone down and you feel like you have seen it al before. As a result you cling to the old and never feel like you grasp the new. The only reasonable thing to is to acknowledge and accept this and try not let it get in your way. Our generation has seen many things before, but at the same time the world has completely changed and it’s led to the…

While this is a nice essay, it also is purely an emotional argument hanging together from assumptions and fallacies. Even if you are right in this instance, just brushing things off with the "you are old" argument will ensure that you end up in some horrible tech debt spaghetti mess in the future. Being critical of the infrastructure you deploy to is a good thing . Because for all the new things that do stick around,…

Well, I fully agree with you. Perhaps the -hate “cloud shit"- remark triggered me a bit. It's just such a 'drown the baby with the bathwater', curmudgeon thing to say. And, imho, it betrays age. It's like my grandfather saying, I hate all this digital stuff, "I will never put email on my phone because with emails come viruses." (Literal thing my father-in-law always claims, and perhaps he's not even wrong, he just stopped using new things, hating and resisting change. He has that right of course. And to be fair with his knowledge level it's perhaps even good to not have email on his Phone. But it's getting more difficult, i.e. he refuses our national Digital ID, making his life a lot harder in the process, especially because he also resists help, too proud). It's good to recognize this in oneself though, imho.
Post reply on HN