Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kubernetes is a red flag signalling premature optimisation

jeremybrown.tech

Post reply on HN
Print view View original

461–470 of 558 posts

Re: Kubernetes is a red flag signalling premature optimisation

#461

Post by nprateem »

mattbillenstein wrote: post #430
nprateem wrote: post #410

Earlier quoted context omitted.

This is optimistic to say the least. I've worked as an SRE for 5 years and apart from the others in the team the devs don't have nearly as much knowledge. There's no way I'd rely on them to fix an outage. And even on a small retainer you'd better hope they retained the knowledge of how all that stuff works if you're only calling on them every now and again.

The idea is devops as _culture_ - I come in as an expert and set it up, then show them how I did it, then run through various disaster recovery scenarios so they learn it and can handle the vast majority of problems. And you'd be surprised how little problems you might have - I've had many VMs with literally years of uptime running without issue. Most people focus on "devops" as a job - and they never bother to teach…

If it was so easy for devs to pick up SRE companies wouldn't be struggling to find good people.

Culture doesn't mean they'll know how to fix at 3am in the morning. The SRE with constant ops exposure is likely to have a much better chance, if only because they (ought to) really know how to debug it.

Re: Kubernetes is a red flag signalling premature optimisation

#462

Post by hllooo »

throwaway787544 wrote: post #389
mountainriver wrote: post #351

Earlier quoted context omitted.

Yeah but k8s isn’t hard at all if you know it, it’s actually substantially easier than a couple web servers and dbserver and provides a whole lot more

Rocket science isn't hard if you know it. Should we all build spaceships to deliver groceries? Good luck finding a few local rocket scientists in a pinch. You can find plenty of auto mechanics though. Cars are cheaper and ubiquitous. Maybe they can't drive to the moon, but they can get most things done. Unless your business is flying to the moon, stick to cars and trucks over spaceships.

lol yes it is

Re: Kubernetes is a red flag signalling premature optimisation

#463

Post by spidermanjones »

danielvaughn wrote: post #287
movedx wrote: post #278

I agree entirely. I like to call what the author is referring to as, "What-If Engineering". It's the science of thinking you'll be Google next week, so you build for that level of scale today. It involves picking extremely complicated, expensive (both in compute and skilled labour) technologies to deploy a Rails app that has two features. And it all boils down to, "But what if..." pre-optimising. It happens at all le…

I've told this story before on HN, but a recent client of mine was on Kubernetes. He had hired an engineer like 5 years ago to build out his backend, and the guy set up about 60 different services to run a 2-3 page note taking web app. Absolute madness. I couldn't help but rewrite the entire thing, and now it's a single 8K SLOC server in App Engine, down from about 70K SLOC.

My most recent job, and the job before that and the job before that all have one thing in common:

Someone convinced the right person to “put stuff on kubernetes” and then booked it for another job/greener pastures with barely any production services, or networking configured.

Thus an opening for a new SRE and once again I find myself looking at a five headed monster of confusing ingress rules, unnamed/unlabeled pods, unclaimed volume specs…

Sigh.

Re: Kubernetes is a red flag signalling premature optimisation

#464

Post by thaneross »

mixedCase wrote: post #411
mirceal wrote: post #377

Earlier quoted context omitted.

> Stop pretending Kubernetes is this humongous infrastructure investment that mandates a full time job to keep up at low scale. It is a full time job. The cost of using something is not just the setup cost. The same way that software development is not just the cost of writing the software.

How? What is keeling over so often on that is not on a simpler setup? I've set up EKS multiple times across different companies with few nodes and everything runs smoothly all of the time. Can you go into detail?

Indeed. Hosted k8s has been low maintenance in my experience, much lower than managing a bunch of VMs. I think the common thread between people who have horror stories using k8s is either pains from self-hosting or teams who adopted it without experience and used the ample rope it gives to hang themselves.

Re: Kubernetes is a red flag signalling premature optimisation

#465

Post by pojzon »

iso1631 wrote: post #286
movedx wrote: post #271

Earlier quoted context omitted.

> deploy elastic and some other helm charts as quickly as possible Bad culture alert! No one needs Elastic "as quickly as possible" unless their business, or the business they work for, is being very poorly run. I would also argue that you might have got it running quickly, but how are you patching it? Maintaining it? Securing it? Backing it up? Have you got a full D/R plan in place? Can you bring it back to life if…

I believe elastic is available as service from AWS and elastic.co, if you need it fast, use that. If you need it long term it may be worthwhile to deploy your own for cost and flexibility purposes

Managing Elastic Cluster is on pair of managing Kubernetes cluster. Its not easy.

Setting it up is easy, but not operations.

Ive been managing multiple clusters on AWS and on Azure.

I would take a managed EKS/OS in a heartbeat if I was a startup.

Re: Kubernetes is a red flag signalling premature optimisation

#466

Post by bhauer »

While I partially agree with the premise, I perceive the situation a bit differently. By not adequately prioritizing performance at the early technology-selection phase of a development project, teams often find themselves forced to prematurely adopt highly-complex deployment systems and other approaches to scale their low-performance system.

In my experience, a well-functioning new team will include performance in their selection process. Doing so allows a project to defer optimizations such as higher-complexity deployment (e.g., cluster orchestration, highly sophisticated caching, and so on) for much longer than those who build on low-performance platforms and frameworks.

Choosing a low-performance platform and framework sets an artificially low performance ceiling. Developers dealing with a low performance ceiling will either instinctively (through received or learned patterns) or reactively (through user complaints, issue reports, troubleshooting, and tuning) deal with performance challenges they should not need to deal with so early in a product's lifespan.

Ironically, some would say that including performance in your technology selection criteria is "premature optimization," but I argue the opposite: considering performance a feature early allows you to defer costly optimizations (such as the matter at hand, Kubernetes deployment), perhaps even indefinitely.

Re: Kubernetes is a red flag signalling premature optimisation

#467

Post by 0xEFF »

ryanbrunner wrote: post #327
engineeringwoke wrote: post #267

Earlier quoted context omitted.

It's healthy to depend on your coworkers for their specific knowledge. The landscape is too large for everyone to know everything, and honestly if I heard someone say this in an interview I would chalk it down to social deficits because this is not how life, businesses, etc. work

I think the key with this though is that it's good when everyone on a team has a working knowledge of something, and one person has expert knowledge. If one person knows everything there is to know, and everyone else knows nothing, you've created a massive dependency on the single person (which in the case of infrastructure code, could easily be a near existential problem for the company).

It’s unrealistic to expect the entire team to know how to build and safely operate IPv6 BGP anycast with an HTTP/2 tls load balancer, authentication and authorization, integrated with a modern observability platform and a zero downtime deployment CI process.

It is realistic to expect a small team to build the same in an industry standard way and hand of a clear, well documented API to the rest of the team.

Bespoke solutions need to deal with this complexity somehow, k8s provides a standard way to do so.

Re: Kubernetes is a red flag signalling premature optimisation

#468

Post by mirceal »

zinclozenge wrote: post #436
mirceal wrote: post #380

Earlier quoted context omitted.

No, it's not. If going for managed services: A load balancer + an asg is stupid simple to setup and it just works.

How do you deploy your code in this scenario, ssh into VMs?

Level 1) you package your service into a zip/rpm/deb/etc and have an agent on the machine that periodically pulls

Level 2) you pack your software into an ami and use the update the asg config. You can periodically "drain" the asg of old instances

Level 3) you deploy your stack again with the new stack having the ami that you've build at level 2 referenced. You start shifting traffic between the old stack and the new stack. You monitor and rollback if something is wrong.

Re: Kubernetes is a red flag signalling premature optimisation

#469

Post by regularfry »

mattbillenstein wrote: post #428
regularfry wrote: post #148

Earlier quoted context omitted.

The question is how much you need to know. You need much less knowledge to run something in Heroku than in k8s; dramatically less in the "onboard a CRUD app" case. I'd argue that running k8s effectively is no less knowledge-intensive than running VMs.

> I'd argue that running k8s effectively is no less knowledge-intensive than running VMs. LOL - are you f'ing kidding? Running VMs is much closer to running your app locally than containers and k8s - it's not that hard to do badly, and only marginally harder to do well.

Having done both, not kidding in the slightest.

Re: Kubernetes is a red flag signalling premature optimisation

#470

Post by 0xCAP »

mirceal wrote: post #468
zinclozenge wrote: post #436

Earlier quoted context omitted.

How do you deploy your code in this scenario, ssh into VMs?

Level 1) you package your service into a zip/rpm/deb/etc and have an agent on the machine that periodically pulls Level 2) you pack your software into an ami and use the update the asg config. You can periodically "drain" the asg of old instances Level 3) you deploy your stack again with the new stack having the ami that you've build at level 2 referenced. You start shifting traffic between the old stack and the new…

Worst solutions I've heard in a while, no offense...
Post reply on HN