Kubernetes is a red flag signalling premature optimisation • Hacker News Board

Re: Kubernetes is a red flag signalling premature optimisation

#481

Post by movedx » Mon, Jul 04, 2022, 9:25 PM UTC

mirceal wrote: post #468

zinclozenge wrote: post #436

Earlier quoted context omitted.

How do you deploy your code in this scenario, ssh into VMs?

Level 1) you package your service into a zip/rpm/deb/etc and have an agent on the machine that periodically pulls Level 2) you pack your software into an ami and use the update the asg config. You can periodically "drain" the asg of old instances Level 3) you deploy your stack again with the new stack having the ami that you've build at level 2 referenced. You start shifting traffic between the old stack and the new…

I find it's easier to use Ansible/Salt/Puppet Bolt and Packer to bake an AMI every night, update the launch template in a DB (which Terraform pulls the value from, thus there is no drift), and auto the ASG. Then you just force a drain.

Now you've got automatic, constantly updating VMs every night if you want them. And a new deployment is just commiting code to master and pushing and that whole pipeline triggers for you.

People like to overcomplicate things, Mirceal. You're on the right path :-)

Re: Kubernetes is a red flag signalling premature optimisation

#482

Post by movedx » Mon, Jul 04, 2022, 9:29 PM UTC

mountainriver wrote: post #353

movedx wrote: post #278

I agree entirely. I like to call what the author is referring to as, "What-If Engineering". It's the science of thinking you'll be Google next week, so you build for that level of scale today. It involves picking extremely complicated, expensive (both in compute and skilled labour) technologies to deploy a Rails app that has two features. And it all boils down to, "But what if..." pre-optimising. It happens at all le…

Sorry but managed k8s is really simple and wildly a better pattern than just running VMs. You don’t need google scale for it to help you, and spinning things up without understanding the maintenance cost is just bad engineering

> Sorry but managed k8s is really simple ...

If you need a service to manage K8s for you, then that's a red flag already (regarding K8s, not you personally.) If the service is so complicated that experienced engineers tell me constantly that only managed K8s is the way to do it, that tells me enough about why it's going to be a rough journey that should probably be avoided with IaaS or PaaS.

> ... and wildly a better pattern than just running VMs.

I've never had an issue running VMs, personally. And when I join a firm and begin helping them, I find I can very quickly and easily come up to speed with their infrastructure when it's based on straight IaaS/SaasS/PaaS. If it's K8s, it's often way more complicated ("Hey! We should template our YAML configs and then use Ansible to produce the Helm charts and then run those against the infra!" - ha ha!)

Re: Kubernetes is a red flag signalling premature optimisation

#483

Post by movedx » Mon, Jul 04, 2022, 9:31 PM UTC

d23 wrote: post #305

movedx wrote: post #278

I agree entirely. I like to call what the author is referring to as, "What-If Engineering". It's the science of thinking you'll be Google next week, so you build for that level of scale today. It involves picking extremely complicated, expensive (both in compute and skilled labour) technologies to deploy a Rails app that has two features. And it all boils down to, "But what if..." pre-optimising. It happens at all le…

The truth is at scale the last thing you want is a nest of unmanaged complexity, so it’s also the wrong instinct there. It’s usually contact with the real world that dictates what needs the extra engineering effort, and trying to do it ahead of time just means you’ll sink time up front and in maintenance on things that didn’t turn out to be your problem.

I think at scale, K8s is a good choice. I run a Discord server with like, 3,400 members now, and some of them are working at mental scale. They've claimed the same as you: K8s at scale is the only way.

I would very likely agree in all cases.

However, they only represent 4-5 users out of all 3,400. And that's the issue - only a small fraction of the industry operates at that scale :-)

Re: Kubernetes is a red flag signalling premature optimisation

#484

Post by ryanbrunner » Mon, Jul 04, 2022, 9:32 PM UTC

0xEFF wrote: post #467

ryanbrunner wrote: post #327

Earlier quoted context omitted.

I think the key with this though is that it's good when everyone on a team has a working knowledge of something, and one person has expert knowledge. If one person knows everything there is to know, and everyone else knows nothing, you've created a massive dependency on the single person (which in the case of infrastructure code, could easily be a near existential problem for the company).

It’s unrealistic to expect the entire team to know how to build and safely operate IPv6 BGP anycast with an HTTP/2 tls load balancer, authentication and authorization, integrated with a modern observability platform and a zero downtime deployment CI process. It is realistic to expect a small team to build the same in an industry standard way and hand of a clear, well documented API to the rest of the team. Bespoke so…

I'd consider knowing how to use the API to configure services and deploy new services to be a good definition of "working knowledge". You're right that everyone doesn't need to know the ins and outs of everything for sure, but if you observe at your company that everyone is relying on "the Kubernetes guy" to do everything related to Kubernetes, you've just re-invented an old-school ops team in an especially brittle way.

Re: Kubernetes is a red flag signalling premature optimisation

#485

Post by movedx » Mon, Jul 04, 2022, 9:32 PM UTC

iso1631 wrote: post #343

movedx wrote: post #285

Earlier quoted context omitted.

I'm not forgetting that fact, I'm simply choosing to ignore such people. They're not really what the industry is about. That's not in the spirit of a healthy society. That's just leeching. Good luck to them, but they're not going to occupy time and space in my mind.

Maybe it's just my organization, but I see the behavior across the corporation far more than I'd like, and inevitably these people move on leaving a complex mess in their wake that long term support staff have to deal with. We seem to mostly manage to avoid that in my department, but we have a very low turnover.

Sounds rough, buddy! Sorry to hear that. I hope you're being well compensated.

Re: Kubernetes is a red flag signalling premature optimisation

#486

Post by movedx » Mon, Jul 04, 2022, 9:33 PM UTC

jjav wrote: post #347

movedx wrote: post #285

Earlier quoted context omitted.

I'm not forgetting that fact, I'm simply choosing to ignore such people. They're not really what the industry is about. That's not in the spirit of a healthy society. That's just leeching. Good luck to them, but they're not going to occupy time and space in my mind.

> They're not really what the industry is about. We'd like that (I'd like that), but resume-driven choices are a very large driver of technology direction, unfortunately. It means those of who want to build something very maintainable and very stable using the most boring (stable, secure) technology possible are often outnumbered.

Sigh. Tell me about it :-/

Re: Kubernetes is a red flag signalling premature optimisation

#487

Post by funstuff007 » Mon, Jul 04, 2022, 9:41 PM UTC

What's good back of envelope break-even for when I should move to K8S from a set of hand-rolled scripts and manual processes to manage a bunch of VMs? It definitely feels like more than 10, but how much more than 10? And do people manage database servers using K8S? Or is it better than hand manage those?

Re: Kubernetes is a red flag signalling premature optimisation

#488

Post by nouveaux » Mon, Jul 04, 2022, 9:50 PM UTC

iasay wrote: post #256

jpgvm wrote: post #251

Earlier quoted context omitted.

99% of people aren't going to use a different CNI plugin to what their managed distribution ships with. Same goes for peeking under the covers of storage plugins, kubelet config, etc. You pay AWS/GCP for that these days and just use the API.

It was AWS who had trouble fixing our CNI issues…

If AWS broke your CNI that caused you problems, it is also possible for AWS to break networking.

I have used Managed K8s for 4 years and literally never had any problems with CNI. My clusters runs with no problems.

Re: Kubernetes is a red flag signalling premature optimisation

#489

Post by nouveaux » Mon, Jul 04, 2022, 9:55 PM UTC

candiddevmike wrote: post #248

sweaver wrote: post #166

For sure, there is power at the cost of agility! I've seen cases where we started off as simply as possible with no k8's. We built the initial product really quickly using a ton of managed services. Whilst it was great to get us going, once we hit "growth" things just didn't scale. (1) The cost of cloud was getting astronomical for us (and growing with each new deployment) and (2) it was totally inflexible (whether t…

I think it's funny that everyone uses/loves k8s ends up building a product to make it easier to use. There are a lot of examples in the comments here. To me, that's enough of a red flag that k8s doesn't understand their users or can't meet them where they're at.

Heroku and friends is literally built to simplify the complexities of VMs. Are you also arguing that deploying to VMs is too complex?

Re: Kubernetes is a red flag signalling premature optimisation

#490

Post by jbverschoor » Mon, Jul 04, 2022, 10:00 PM UTC

osigurdson wrote: post #298

movedx wrote: post #278

I agree entirely. I like to call what the author is referring to as, "What-If Engineering". It's the science of thinking you'll be Google next week, so you build for that level of scale today. It involves picking extremely complicated, expensive (both in compute and skilled labour) technologies to deploy a Rails app that has two features. And it all boils down to, "But what if..." pre-optimising. It happens at all le…

>> Helm and the likes make it possible to spin up these kinds of solutions in a heart beat Genuine question, why is this bad? Is it because k8s can spin it up but becomes unreliable later? I think the industry wants something like k8s - define a deployment in a file and have that work across cloud providers and even on premise. Why can't we have that? It's just machines on a network after all. Maybe k8s itself is jus…

We had J2EE almost 30 years ago. 1 file which described everything and contained everything