Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

blog.stackademic.com

Post reply on HN
Print view View original

171–180 of 216 posts

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#171

Post by notnmeyer »

InDubioProRubio wrote: post #163
karmajunkie wrote: post #156

Earlier quoted context omitted.

that tends to be the take on most “k8s is too complex” articles, at least the ones i’ve seen. yes, it’s complex, but it’s simpler than running true high availability setups without something like it to standardize the processes and components needed. what i want to see is a before and after postmortem on teams that dropped it and compare their numbers like outages to get at the whole truth of their experience.

Complexity is a puzzle and attracts a certain kind of easy bored dev, who also has that rockstar flair, selling it to management - then quitting (cause bored) leaving a group of wizard-prophet-whorshippers to pray to the k8 goddess at the monolith circle at night. And you can not admit as management, that you went all in on a guru and a cult.

…i’m sorry, what? k8s can be as simple or complex as you make it.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#172

Post by pickle-wizard »

karmarepellent wrote: post #62
oblio wrote: post #59

Earlier quoted context omitted.

Kubernetes upgrades? Don't those risk bricking everything with just 1 environment?

In theory: absolutely. This is just anecdata and you are welcome to challenge me on it, but I have never had a problem upgrading Kubernetes itself. As long as you trail one version behind the latest to ensure critical bugs are fixed before you risk to run into them yourself, I think you are good. Edit: To expand on it a little bit. I think there is always a real, theoretical risk that must be taken into account when…

At my last job we had a Kubernetes upgrade go so wrong we ended up having to blow away the cluster and redeploy everything. Even a restore of the etcd backup didn't work. I couldn't tell you exactly what went wrong, as I wasn't the one that did the upgrade. I wasn't around of the RCA on this one. As the fallout was straw that broke the camels back, I ended up quitting to take a sabbatical.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#173

Post by withinboredom »

InDubioProRubio wrote: post #163
karmajunkie wrote: post #156

Earlier quoted context omitted.

that tends to be the take on most “k8s is too complex” articles, at least the ones i’ve seen. yes, it’s complex, but it’s simpler than running true high availability setups without something like it to standardize the processes and components needed. what i want to see is a before and after postmortem on teams that dropped it and compare their numbers like outages to get at the whole truth of their experience.

Complexity is a puzzle and attracts a certain kind of easy bored dev, who also has that rockstar flair, selling it to management - then quitting (cause bored) leaving a group of wizard-prophet-whorshippers to pray to the k8 goddess at the monolith circle at night. And you can not admit as management, that you went all in on a guru and a cult.

Then they hire a different cult leader, one that can clean up the mess and simplify it for the cult that was left behind. The old cult will question their every motive, hinder them with questions about how they could ever make it simpler. Eventually, once the last old architecture is turned off; they will see the errors of their ways. This new rock star heads off to the next complicated project.

A new leader arrives and says, “we could optimize it by…”

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#174

Post by withinboredom »

Elidrake24 wrote: post #134
anthonybsd wrote: post #129

Earlier quoted context omitted.

>Often you'd need production, staging, test, development, something like that. Normally in K8s, segregating environments is done via namespaces, not clusters (unless there are some very specific resource constraints).

Which in many cases would break SOC2 compliance (co-mingling of development and customer resources), and even goes against the basic advice offered in the K8s manual. Beyond that, this limits your ability to test Control Plane upgrades against your stack, though that has generally been very stable in my experience. To be clear I'm not defending the 47 Cluster setup of the OP, just the practice of separating Developme…

Why would you commingle development and customer resources? A k8s cluster is just a control plane, that specifically controls where things are running, and if you specify they can’t share resources, that’s the end of that.

If you say they share the same control plane is commingling… then what do you think a cloud console is? And if you are using different accounts there… then I hope you are using dedicated resources for absolutely everything in prod (can’t imagine what you’d pay for dedicated s3, sqs) because god forbid those two accounts end up on the same machine. Heh, you are probably violating compliance and didn’t even know it!

Sigh. I digress.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#175

Post by zeroc8 »

It's the same story over and over again. Nobody gets fired for choosing AWS or Azure. Clueless managers and resume driven developers, a terrible combination. The good thing is that this leaves a lot of room for improvement for small companies, who can out compete larger ones by just not making those dumb choices.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#176

Post by withinboredom »

bdndndndbve wrote: post #140
anthonybsd wrote: post #129

Earlier quoted context omitted.

>Often you'd need production, staging, test, development, something like that. Normally in K8s, segregating environments is done via namespaces, not clusters (unless there are some very specific resource constraints).

I would want to have at least dev + prod clusters, sometimes people want to test controllers or they have badly behaved workloads that k8s doesn't isolate well (making lots of giant etcd objects). You can also test k8s version upgrades in non-prod. That said it sounds like these people just made a cluster per service which adds a ton of complexity and loses all the benefits of k8s.

In this case, I use a script to spin up another production cluster, perform my changes, and send some traffic to it. If everything looks good, we shift over all traffic to the new cluster and shutdown the old one. Easy peasy. Have you turned your pets into cattle only to create a pet ranch?

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#177

Post by processunknown »

jppittma wrote: post #143
this_user wrote: post #130

Earlier quoted context omitted.

It's just a matter of time before someone releases an orchestration layer for k8s clusters so the absurd Rube Goldberg machine that is modern devops stacks can grow even more complex.

https://istio.io/latest/

Istio is not a k8s orchestration layer though

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#178

Post by withinboredom »

AtlasBarfed wrote: post #127
mrweasel wrote: post #96

Earlier quoted context omitted.

> Isn’t the point of kubernetes that you can run your entire infra in a single cluster I've never seen that, but yes 47 seems like a lot. Often you'd need production, staging, test, development, something like that. Then you'd add an additional cluster for running auxiliary service, this is services that has special network access or are not related to you "main product". Maybe a few of these. Still there's a long wa…

K8s requires a flat plane addressability model across all containers, meaning anyone can see and call anyone else? I can see security teams getting uppity about that. Also budgetary and org boundaries, cloud providers, disaster recovery/hot spares/redundancy/AB hotswap, avoid single tank point of failure.

This… sounds remarkably like the problems kubernetes solves.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#179

Post by AtlasBarfed »

withinboredom wrote: post #178
AtlasBarfed wrote: post #127

Earlier quoted context omitted.

K8s requires a flat plane addressability model across all containers, meaning anyone can see and call anyone else? I can see security teams getting uppity about that. Also budgetary and org boundaries, cloud providers, disaster recovery/hot spares/redundancy/AB hotswap, avoid single tank point of failure.

This… sounds remarkably like the problems kubernetes solves.

single tank point of failure should be

single YAML point of failure

mobile autocorrect is super "helpful"

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#180

Post by alexyz12 »

zeroc8 wrote: post #175

It's the same story over and over again. Nobody gets fired for choosing AWS or Azure. Clueless managers and resume driven developers, a terrible combination. The good thing is that this leaves a lot of room for improvement for small companies, who can out compete larger ones by just not making those dumb choices.

but does improving really help these small companies in the way it matters? If the cost of the infrastructure is apparently not important to the needs of the business..
Post reply on HN