Kubernetes is a red flag signalling premature optimisation
jeremybrown.tech
Kubernetes is a red flag signalling premature optimisation
1–10 of 558 posts
Re: Kubernetes is a red flag signalling premature optimisation
#2[OP here] It feels bizarre saying this, having spent so much of my life advocating for and selling a distribution of Kubernetes and consulting services to help folks get the most of out it, but here goes! YOU probably shouldn't use Kubernetes and a bunch of other "cool" things for your product.
Most folks building software at startups and scale-ups should avoid Kubernetes and other premature optimisations. If your company uses Kubernetes, you are likely expending energy on something that doesn't take you towards your mission. You have probably fallen into the trap of premature optimisation.
Please don't take this post to be only aimed against Kubernetes. It is not. I am directing this post at every possible bit of premature optimisation engineers make in the course of building software.
Re: Kubernetes is a red flag signalling premature optimisation
#3In the limit, there are some startups that could run production on a single Linux host - I recently helped one get off Heroku and their spend went from ~$1k/mo to ~$50/mo and it made debugging and figuring out performance issues so much easier than what they were doing previously...
Re: Kubernetes is a red flag signalling premature optimisation
#4Having worked with Kubernetes, it's great - I think even smaller setups can benefit from it and its approach (especially when using a managed provider).
But for startups and/or simpler setups, ECS/Lambda is so much less work, while usually being powerful enough.
Re: Kubernetes is a red flag signalling premature optimisation
#5[OP here] It feels bizarre saying this, having spent so much of my life advocating for and selling a distribution of Kubernetes and consulting services to help folks get the most of out it, but here goes! YOU probably shouldn't use Kubernetes and a bunch of other "cool" things for your product. Most folks building software at startups and scale-ups should avoid Kubernetes and other premature optimisations. If your co…
Running a self-hosted Kubernetes is indeed ... questionable, but a managed Kubernetes? That's a pretty sane thing to do IMO. The alternatives are running your Docker containers either manually on some EC2 or other bare-metal server which is a nightmare to do deployments, or using something like Elastic Beanstalk which is even worse.
For me at least, Kubernetes has become something like an universal standard: if you're already running Docker or other containerization, "using Kubernetes" is nothing more than a clear, "single source of truth" documentation on what infrastructure your application expects. Certainly beats that Confluence document from 2018, last updated 2019, on how the production server is set up that doesn't even match up with what was reality in 2020 much less today.
Re: Kubernetes is a red flag signalling premature optimisation
#6[OP here] It feels bizarre saying this, having spent so much of my life advocating for and selling a distribution of Kubernetes and consulting services to help folks get the most of out it, but here goes! YOU probably shouldn't use Kubernetes and a bunch of other "cool" things for your product. Most folks building software at startups and scale-ups should avoid Kubernetes and other premature optimisations. If your co…
Doesn't it depend what staff you have? I would have agreed with this in 2018 but the world has moved on.
If you have a bunch of people who know it, you can deploy a cluster into gcloud or aws with a few clicks or lines of IaC.
Would I recommend a startup team learn kube while trying to ship a product? No.
Would I think it's a red flag if a team who already know it choose it as their preferred platform? Also no.
Re: Kubernetes is a red flag signalling premature optimisation
#7Our startup has a webapp that acts as a UI for a machine learning application. We have two different types of heavy workloads (ML and something else). The workers for these run on Kubernetes, which makes them easy to scale (which we do a lot, automatically). The app itself doesn't run on Kubernetes yet (simple VM). It would be better if it did, though! We keep building a lot of functionality ourselves (e.g. deployment without downtime) that would be easier with K8s. I think what you really want to avoid is Microservices, not K8s. K8s gives you many advantages, such as the entire config in Git, containerized workloads, deploy management, etc., which sooner or later, you'll find yourself wanting. Depending on how you get started, the learning curve can be steep, and some of the tools are not mature but supposedly used by everyone (ArgoCD for deployment?).
Re: Kubernetes is a red flag signalling premature optimisation
#8In the limit, there are some startups that could run production on a single Linux host - I recently helped one get off Heroku and their spend went from ~$1k/mo to ~$50/mo and it made debugging and figuring out performance issues so much easier than what they were doing previously...
I'm very much with you on this, but I do understand that it's one of those things that is just not feasible when your team has no sysadmin/devops experience.
You were able to do it, but what happens to them when you're not around? Does their team have the required experience to handle it? That's the difference in cost. It's like DYI - yes, if I have all the skills and experience I can do everything myself incredibly cheaply, but... I don't. So I gotta pay.
But if your team does have the skills and experience, it's definitely worth looking into.
I do think people deeply underestimate what can be achieved with a single (or two, if you need a standby) dedicated Linux server these days. A single server can easily go up to 2TB of RAM and 128+ cores. Long before you ever get to a scale where that's a limitation, you'll have more than enough resources to figure things out.
Re: Kubernetes is a red flag signalling premature optimisation
#9[OP here] It feels bizarre saying this, having spent so much of my life advocating for and selling a distribution of Kubernetes and consulting services to help folks get the most of out it, but here goes! YOU probably shouldn't use Kubernetes and a bunch of other "cool" things for your product. Most folks building software at startups and scale-ups should avoid Kubernetes and other premature optimisations. If your co…
You can get away without an orchestrator right up until about when your ARR hits $10mm
Hiring people with k8s experience in 2022 is not a difficult task, it's not an obscure technology anymore, and when your initial crop of devops decides to leave, the new guys coming in can scan your deployments and namespaces and pretty much hit the ground running within 48-72 hours. That's a big, important part of running a business. Being able to hire people with the right skill set, and being able to keep the lights on.
Bespoke systems built on top of EC2 require time, extra documentation and a steep learning curve, not to mention the fact that the bespoke system probably isn't getting security audits or being built to modern standards or best practices, tooling isn't being kept up to date. I can build a mechanical wristwatch in my garage, but if I had full access to the Rolex factory floor for free, I'd probably take that option.
Re: Kubernetes is a red flag signalling premature optimisation
#10Our startup has a webapp that acts as a UI for a machine learning application. We have two different types of heavy workloads (ML and something else). The workers for these run on Kubernetes, which makes them easy to scale (which we do a lot, automatically). The app itself doesn't run on Kubernetes yet (simple VM). It would be better if it did, though! We keep building a lot of functionality ourselves (e.g. deploymen…
What is it giving you over an ec2 auto scaling group or ECS/Fargate?
Both can scale as much as you like, your config can live as cdk/cloudformation/terraform code?