Earlier quoted context omitted.
Start-ups that don't need to scale will quickly go away, because how else are you going to make a profit? How have you been going since 2005 and still not understand the economics of software?
CPUs are ~300x more powerful and storage offers ~10,000x more IOPS than 2005 hardware. More efficient server code exists today. You can scale very far on one server. If you were bootstrapping a startup, you could probably plan to use a pair of gaming PCs until at least the first 1-10M users.
I Didn't Need Kubernetes, and You Probably Don't Either
281–290 of 436 posts
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#282Earlier quoted context omitted.
Kubernetes starts sucking at first sight and is kind of the sign of the ailment of modern times. Let me try to explain: First, you encounter the biggest impedance mismatch between cloud and on prem: Kubernetes works with pods, while AWS works with instances as the unit of useful work, so they must map to each other, right? Wrong, first each instance needs to run a Kubernetes node, which duplicates the management infr…
It really, really wants a higher-level abstraction layer over the top of it. I can see how you'd build something heroku-like with it, but exposing app developers to it just seems cruel.
That's what many teams end up half-arsing without realising they're attempting to build a PaaS.
They adopt K8S thinking it's almost 90% a PaaS. It's not.
They continue hiring, building a DevOps team just to handle K8S infrastructure things, then a Platform team to build the PaaS on top of it.
Then because so many people have jobs, nobody at this point wants to make an argument that perhaps using an actual PaaS might make sense. Not to mention "the sunk cost" of the DIY PaaS.
Then on top of that, realising they've built a platform mostly designed for microservices, everything then must become a microservice. 'Monolith' is a banned word.
Argh
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#283Wait... so you're saying, I can rent a couple of beefy boxes instead, and live a life where I don't need to know what TolerateDuringExecutionButNotDuringScheduling does?!
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#284Earlier quoted context omitted.
Those "necessary" add-ons and sidecars are out of control, but its the people problem. I'm part of the infra team and we manage just couple of k8s clusters, but those are quite big and have very high traffic load. The k8s + terraform code is simple, with no hacks, reliable and easy to upgrade. Our devs love it, we love it too and all of this makes my job pleasant and as little stressful as possible. But we recently h…
> So its not inexperienced engineers wanting newest hotness because they have no idea how to solve stuff with the tools they have, its sometimes senior engineers trying to justify their salary, "seniority" by buying into complexity as they try to make themselves irreplaceable. The grass is always greener where you water it. They joined your company because the grass was greener there than anywhere else they could get…
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#285Re: I Didn't Need Kubernetes, and You Probably Don't Either
#286Earlier quoted context omitted.
Absolutely, a reasonably sophisticated scalable app platform looks like a half-baked and undocumented reimagination of Kubernetes. Admittedly: The ecosystem is huge and less is more in most cases, but the foundation is cohesive and sane. Would love to read the k8s for greybeards book.
> Absolutely, a reasonably sophisticated scalable app platform looks like a half-baked and undocumented reimagination of Kubernetes. Or maybe Kubernetes looks like a committee designed, everything and the kitchen sink, over-engineered, second system effect, second system effect suffering, YAGNI P.O.S., that only the kind of "enterprise" mindset that really enjoyed J2EE in 2004 and XML/SOAP vs JSON/REST would love...
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#287I just use a single k3s install on a single bare metal from Hetzner or OVH, works like charm, very clean deployments, much more stable than docker-compose and 1/10 of the cost of AWS or similar.
Do you have a write-up about this that you have to share, even if it's someone else's? I'd be curious to try this out.
I have an infrastructure layer that I apply to all clusters that includes things like cert-manager, an ingress controller and associated secrets. This is all cluster-independent stuff. Then some cluster-dependent stuff like storage controllers etc. I use flux to keep this stuff under version control and automatically reconciled.
From there you just deploy your app with standard manifests or however you want to do it (helm, kubectl, flux, whatever).
It all works wonderfully. The one downside is all the various controllers do eat up a fair amount of CPU cycles and memory. But it's not too bad.
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#288Earlier quoted context omitted.
I agree, but what pisses me the most is that today higher level abstractions (like cloud, spring boost,...) are hiding lower level functionality so well, that you are literally forced to use obnoxious amounts of time to study documentation (if you are in luck and it is well written), while everything is decorated with new naming of known concepts that was invented by people who didn't know that the concept already ex…
The real benefits of LLMs is to give me a good summary of these documents. Still, using these abstractions is probably not worth it.
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#289If you want to skip devops work of setting up cluster, go K8s managed. All major cloud providers give you that option.
So, you replaced opensource, standard cloud solution such as K8s with proprietary cloud platform by Google. I mean, good for you but for many of us this sounds backwards.
As a bootstrapped founder, going with K8s was the best decision ever: deployment, scaling, custom resources, jobs, cron, etc, all that stuff comes included and it costs writing a YAML file.
Do you have to learn something to use K8s. Yes, like with everything.
Is it difficult? No more difficult than any other solution out there.
Re: I Didn't Need Kubernetes, and You Probably Don't Either
#290I’ve come to the conclusion that I hate “cloud shit”, and a small part of me is convinced that literally no one actually likes it, and everyone is playing a joke on me. I have set up about a dozen rack mount servers in my life, installing basically every flavor of Unix and Linux and message busses under the sun in the process, but I still get confused by all the Kubectl commands and GCP integration with it. I might j…
1. Cloud for me is a lot better than what we had before: Before i had to create a ticket for our internal it department, have huge cross-charges (like 500$ for a server, instead of 50), had to wait for a few weeks and than get lectured that installing basic default tools on that suse based server would take a week and add additional cross-charges onto it.
Their reasoning? Oh we do backup and upgrades...
With cloud, i click a server in a Minute for less money, i upgrade it myself and have snapshots as a basic backup workflow which actually is reliable and works.
Then k8s came along and let me be clear: my k8s setup is big, so its definitly worth it but tx to my experience with a bigger k8s setup, my very small ones are also working very very well. I get, out of the box, HA, network policies, snapshotting through my selected storage setup, Infrastructure as code etc.
Instead of having shitty shell scripts, ansible setup and co, i only write a little bit of yaml, check it into my git system and roll it out with ansible. Absolut no brainer.
And the auto healing solved real issues: Out of memroy? just restart the pod. Out of disk? Just recreate it. Logging and metrics just works out of the box thanks to the prometheus based monitoring stack.
Starting with one server, yeah why not but you know you are not doing it right if you are the only one who can set it up and recover it. But if you don't have anyone with expertise, i would also not just start with k8s.
If my startup is a pure application + db thingy, i would go with any app platform out there. But we have real servers because we do stuff with data and need sometimes performance, performance is expensive if you run it in the cloud.