Nice solution. Just use vendor lock-in, so you can't use your own shit later.
I Didn't Need Kubernetes, and You Probably Don't Either
211–220 of 436 posts
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#212I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…
All of these anecdotes seem to come from people who don’t bother to try to learn kubernetes.
> YAML files, and then spend a day fixing them by copy-pasting increasingly-convoluted things on stackexchange.
This is terrible behavior. Its not any different from yanking out pam modules because you’re getting SSH auth failures caused by a bad permission on an SSH key.
> If I get to tens of millions of users, maybe I’ll worry about it then.
K8s isn’t there for 10s of millions of users. It’s there so you’re not dependent on some bespoke VM state. It also allows you to do code review on infra changes like port numbers being exposed, etc.
Separately, your VM likely isn’t coming from any standard build pipeline so now a vulnerability patch is a login to the machine and an update, which hopefully leaves it in the same state as VMs created new…
Oh, and assuming you don’t want to take downtime on every update, you’ll want a few replicas and load balancing across them (or active/passive HA at a minimum). Good luck representing that as reviewable code as well if you are running VMs.
The people that don’t understand the value prop of infra as code orchestration systems like k8s tend to work in environments where “maintenance downtime” is acceptable and there are only one or two people that actually adjust the configurations.
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#213> Kubernetes comes with substantial infrastructure costs that go beyond DevOps and management time. The high cost arises from needing to provision a bare-bones cluster with redundant management nodes. That's your problem right there. You really don't want to be setting up and managing a cluster from scratch for anything less than a datacenter-scale operation. If you are already on a cloud provider just use their mana…
For most small setups, the cost of running an empty kubernetes cluster (managed) is typically higher than setting up a db, a couple of vms and a loadbalancer, which goes a long way for running a simple service. Add some buckets, a CDN and you are pretty much good to go.
If you need dedicated people just to stay on top of running your services, you have a problem that's costing you hundreds of thousands per year. There's a lot of fun and easy stuff you can do with that kind of money. This is a pattern I see with a lot of teams that get sucked into using Kubernetes, micro services, terraform, etc. Once you need a few people just to stay on top of the complexity that comes from that, you are already spending a lot. I tend to keep things simple on my own projects because any amount of time I spend on that, I'm not spending on more valuable work like adding features, fixing bugs, etc.
Of course it's not black and white and there's always a trade off between over and under engineering. But a lot of teams default to over engineering simply by using Kubernetes from day one. You don't actually need to. There's nothing wrong with a monolith running on two simple vms with a load balancer in front of it. Worked fine twenty years ago and it is still perfectly valid. And it's dead easy to setup and manage in most popular cloud environments. If you use some kind of scaling group, it will scale just fine.
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#214By the way how much malice can we attribute to cloud vendors?
>Moreover, Kubernetes’s slow autoscaling meant I had to over-provision services to ensure availability, paying for unused resources rather than scaling based on demand.
A typical Linux instance on AWS starts up in about 8 seconds from the asking to start to command line, so lets double that. You could start up and shut down instances in 15 seconds.
Why the hell do you need to overprovision instances then, or leave empty ones running?
I don't see any other reasonable explanation than its in the best interest of cloud vendors to not have short lived instances for the purposes of load balancing, as well as make you consume as much CPU time as possible, even if you don't need it.
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#215I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…
I'm always kind of blown away by experiences like this. Admittedly, I've been using Kubernetes since the early days and I manage an Infra team that operates a couple thousand self-managed Kubernetes clusters so... expert blindness at work. Before that I did everything from golden images to pushing changes via rsync and kicking a script to deploy. Maybe it's because I adopted early and have grown with the technology i…
K8S is a beast and the ecosystem is wild. Newcomer won't know how to proceed to keep things simple, while still understanding everything that is being used.
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#216The worst thing about the whole kubernetes-cult and the like is that IaC and CM was supposed to help us reduce configuration, to make it less prone to failure and easier to manage.
But the truth is that we ran service based architecture, network meshes and containers with bash just fine before cloud everything, usually with less effort than it is to do literally anything today. Sure you had to know how to set up network bonding and how to tune your systems.
Very, very few people and businesses _needs_ kubernetes or it's cousins. Most just need a decent system administrator.
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#217I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…
I'm always kind of blown away by experiences like this. Admittedly, I've been using Kubernetes since the early days and I manage an Infra team that operates a couple thousand self-managed Kubernetes clusters so... expert blindness at work. Before that I did everything from golden images to pushing changes via rsync and kicking a script to deploy. Maybe it's because I adopted early and have grown with the technology i…
My problem is the brittleness.
Sure, I am playing with fire (k3s, bare metal, cilium, direct assigned IP to Ingresses), but a few weeks ago on one cluster suddenly something stopped working in the external IP -> internal cluster IP network path. (And after a complete restart things got worse. Oops. Well okay time to test backups.)
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#218I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…
All of these anecdotes seem to come from people who don’t bother to try to learn kubernetes. > YAML files, and then spend a day fixing them by copy-pasting increasingly-convoluted things on stackexchange. This is terrible behavior. Its not any different from yanking out pam modules because you’re getting SSH auth failures caused by a bad permission on an SSH key. > If I get to tens of millions of users, maybe I’ll wo…
Just because you're using VMs doesn't mean you're now dealing with state.
It's 100% possible to have stateless VMs running in an auto-scaling instance group (in GCP speak, I forget what AWS calls them)
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#219Earlier quoted context omitted.
I'm always kind of blown away by experiences like this. Admittedly, I've been using Kubernetes since the early days and I manage an Infra team that operates a couple thousand self-managed Kubernetes clusters so... expert blindness at work. Before that I did everything from golden images to pushing changes via rsync and kicking a script to deploy. Maybe it's because I adopted early and have grown with the technology i…
>expert blindness at work. >It's not that complicated if you limit yourself to the core stuff. Isn't this the core problem with a lot of technologies. There's a right way to use it, but most ways are wrong. An expert will not look left and right anymore, but to anyone entering the technology with fresh eyes it's a field with abundance of landmines to navigate around. It's simply bad UX and documentation. It could pro…
This and a terminology rug pull. You wanted to upload a script and install some deps? Here’s your provisioning genuination frobnicator tutorial, at the end of which you’ll learn how to maintain the coalescing encabulation for your appliance unit schema, which is needed for automatic upload. It always feels like thousands times bigger complexity (just in this part!) than your whole project.
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#220I still just use VPS devices and some colocated hardware.
I'm constantly told "I am not the typical customer, most do use K8s".
Side projects and volunteer things I run include 470 websites serving about 320K registered users and about 500K-1M monthly guest users... on about 15 VPS devices or small servers.
It's just classic dynamic websites, HTML is the output of a server (no single page web apps), lots of caching for guest users, reasonable complexity (load balancers ahead of web front ends ahead of API back ends ahead of databases).
Things that look like microservices exist... it's just API calls, and they come back through the front door, it's easy to reason about.
Monitoring is mostly Prometheus node exporter, it turns out that CPU + Memory + Disk IOPS + Network IOPS is +90% of what you need... some HTTP logs or profiles are the last 10%.
It's just simple... this is run in my spare time, effort is under an hour per month. (and no it's not monetised, not everything has to be)