Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

blog.stackademic.com

Post reply on HN
Print view View original

181–190 of 216 posts

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#181

Post by jeffwask »

rdsubhas wrote: post #128

I wouldn't trust the management of this team for anything. They appear totally incompetent in both management and basic half-brain analytical skills. Who in the heck creates a cluster per service per cloud provider, duplicate all the supporting services around it, burn money and sanity in a pit, and blame the tool. Literally every single decision they listed was to use any of the given tools in the absolute worst, in…

They lost me at, "And for the first time in two years, our DevOps team took uninterrupted vacations." which is an abject failure of leadership.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#182

Post by Aperocky »

whynotmaybe wrote: post #12
risson wrote: post #3

So they made bad architecture decisions, blamed it on Kubernetes for some reason, and then decided to rebuild everything from scratch. Solid. The takeaway being what? Don't make bad decisions?

My personal takeaway, when it fails and you can't blame anyone for any reason, blame a tool. You'll get the whole team helping you when replacing the tool and setting up a better solution. If you blame anyone, people will start to be extra cautious and won't take any initiative. But don't overuse it, if you always blame the tool, you'll end up like my ex colleague "Steve" where every failure was Microsoft's fault.

Your ex-colleague may not be factually correct, but I agree with him in spirit.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#183

Post by levifig »

figassis wrote: post #18

Why exactly did they have 47 clusters? One thing I noticed (maybe because I’m not at that scale) is that companies are running 1+ clusters per application. Isn’t the point of kubernetes that you can run your entire infra in a single cluster, and at most you’d need a second cluster for redundancy, and you can spread nodes across regions and AZs and even clouds? I think the bottleneck is networking and how much crossta…

To answer your question directly: yes, that's the point. You may have different clusters for different logical purposes but, yes: less clusters, more node groups is a better practice.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#184

Post by mmcnl »

figassis wrote: post #18

Why exactly did they have 47 clusters? One thing I noticed (maybe because I’m not at that scale) is that companies are running 1+ clusters per application. Isn’t the point of kubernetes that you can run your entire infra in a single cluster, and at most you’d need a second cluster for redundancy, and you can spread nodes across regions and AZs and even clouds? I think the bottleneck is networking and how much crossta…

In my company (large corporate) we only have 3 clusters: dev/acc/prod. Everything runs on it. I love it.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#185

Post by mmcnl »

anthonybsd wrote: post #129
mrweasel wrote: post #96

Earlier quoted context omitted.

> Isn’t the point of kubernetes that you can run your entire infra in a single cluster I've never seen that, but yes 47 seems like a lot. Often you'd need production, staging, test, development, something like that. Then you'd add an additional cluster for running auxiliary service, this is services that has special network access or are not related to you "main product". Maybe a few of these. Still there's a long wa…

>Often you'd need production, staging, test, development, something like that. Normally in K8s, segregating environments is done via namespaces, not clusters (unless there are some very specific resource constraints).

Sometimes there are requirements to separate clusters on the network level.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#187

Post by _bare_metal »

marcinzm wrote: post #150
_bare_metal wrote: post #126

Earlier quoted context omitted.

To expand on this, I run BareMetalSavings.com[0] and the most common cause of people staying with the cloud is it's very hard for them to maintain their own K8S cluster(s), which they want to keep because they're great for any non-ops developer. So those savings are possible only if your devs are willing to leave the lock in of comfort [0]: https://BareMetalSavings.com

Cloud isn't about comfort lock in but dev efficiency.

Not every use case is more efficient on the cloud

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#188

Post by UltraSane »

englishspot wrote: post #141
this_user wrote: post #130

Earlier quoted context omitted.

It's just a matter of time before someone releases an orchestration layer for k8s clusters so the absurd Rube Goldberg machine that is modern devops stacks can grow even more complex.

karmada? https://karmada.io/

Is it ever a good idea to actually use this?

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#189

Post by bdcravens »

chris_wot wrote: post #132

Would have loved to know more about this, but I sure as heck am not going to pay to find out.

Basically they went AWS-native, with ECS being the biggest part of that. I'm currently trying to move our own stack into a simpler architecture, but I can wholeheartedly recommend ECS as a Kubernetes alternative, giving you 80% of the functionality for 20% of the effort.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#190

Post by lenkite »

mdaniel wrote: post #159
lenkite wrote: post #154

Earlier quoted context omitted.

Most important reason is the ridiculous etcd limit of 8Gb. That alone is the reason for most k8s cluster splits.

I hate etcd probably more than most, but that 8Gb seems to just be a warning, unless you have information otherwise https://etcd.io/docs/v3.5/dev-guide/limit/#storage-size-limi... I'll take this opportunity to once again bitch and moan that Kubernetes just fucking refuses to allow the KV store to be pluggable, unlike damn near everything else in their world, because they think that's funny or something

It isn't a mere warning. It is strongly recommended as the upper limit.

https://www.perfectscale.io/blog/etcd-8gb

https://github.com/etcd-io/etcd/issues/9771

And yes, I agree not allowing a pluggable replacement is really stupid.

Post reply on HN