Live data from Hacker News

I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

blog.stackademic.com

21–30 of 216 posts

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#21
I feel like OP would've been much better off if they just reworked their cluster into something sensible instead of abandoning K8s completely.

I've worked on both ECS and K8s, and K8s is much better. All of the problems they listed were poor design decisions, not k8s limitations.

- 47 Clusters: This is insane. They ack it in the post, but they could've reworked this.

- Multi-cloud: It's now not possible with ECS, but they could've limited complexity with just single-cloud k8s.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#22
post #17

Oh boy. Please, please stop using Medium for anything. I have lost count of how many potentially interesting or informative articles are published behind the Medium sign-in wall. At least for me, if you aren't publishing blog articles in public then what's the point of me trying to read them.

word

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#23
post #21

I feel like OP would've been much better off if they just reworked their cluster into something sensible instead of abandoning K8s completely. I've worked on both ECS and K8s, and K8s is much better. All of the problems they listed were poor design decisions, not k8s limitations. - 47 Clusters: This is insane. They ack it in the post, but they could've reworked this. - Multi-cloud: It's now not possible with ECS, but…

47 clusters is major "we didn't know what we were doing"

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#24
post #17

Oh boy. Please, please stop using Medium for anything. I have lost count of how many potentially interesting or informative articles are published behind the Medium sign-in wall. At least for me, if you aren't publishing blog articles in public then what's the point of me trying to read them.

    > "Create an account to read the full story."
Why is this required?

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#25
post #17

Oh boy. Please, please stop using Medium for anything. I have lost count of how many potentially interesting or informative articles are published behind the Medium sign-in wall. At least for me, if you aren't publishing blog articles in public then what's the point of me trying to read them.

> "Create an account to read the full story." Why is this required?

[dead]

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#26
post #21

I feel like OP would've been much better off if they just reworked their cluster into something sensible instead of abandoning K8s completely. I've worked on both ECS and K8s, and K8s is much better. All of the problems they listed were poor design decisions, not k8s limitations. - 47 Clusters: This is insane. They ack it in the post, but they could've reworked this. - Multi-cloud: It's now not possible with ECS, but…

47 clusters is major "we didn't know what we were doing"

At my last gig we were in the process of sunsetting 200+ clusters. We allowed app teams to request and provision their own cluster. That 3 year experiment ended with a migration down to 24ish clusters (~2 clusters per datacenter)

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#27

Why exactly did they have 47 clusters? One thing I noticed (maybe because I’m not at that scale) is that companies are running 1+ clusters per application. Isn’t the point of kubernetes that you can run your entire infra in a single cluster, and at most you’d need a second cluster for redundancy, and you can spread nodes across regions and AZs and even clouds? I think the bottleneck is networking and how much crossta…

There are genuine reasons for running multiple clusters. It helps to sometimes keep stateful (databases generally) workloads on one cluster, have another for stateless workloads etc. Sometimes customers demand complete isolation so they get their own cluster (although somehow its ok that the nodes are still VMs that are probably running on shared nodes… these requirements can be arbitrary sometimes).

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#28
post #27

Why exactly did they have 47 clusters? One thing I noticed (maybe because I’m not at that scale) is that companies are running 1+ clusters per application. Isn’t the point of kubernetes that you can run your entire infra in a single cluster, and at most you’d need a second cluster for redundancy, and you can spread nodes across regions and AZs and even clouds? I think the bottleneck is networking and how much crossta…

There are genuine reasons for running multiple clusters. It helps to sometimes keep stateful (databases generally) workloads on one cluster, have another for stateless workloads etc. Sometimes customers demand complete isolation so they get their own cluster (although somehow its ok that the nodes are still VMs that are probably running on shared nodes… these requirements can be arbitrary sometimes).

Honest question - why would you want stateful workloads in a separate cluster from stateless? Why not just use a namespace?

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#29
post #27

Why exactly did they have 47 clusters? One thing I noticed (maybe because I’m not at that scale) is that companies are running 1+ clusters per application. Isn’t the point of kubernetes that you can run your entire infra in a single cluster, and at most you’d need a second cluster for redundancy, and you can spread nodes across regions and AZs and even clouds? I think the bottleneck is networking and how much crossta…

There are genuine reasons for running multiple clusters. It helps to sometimes keep stateful (databases generally) workloads on one cluster, have another for stateless workloads etc. Sometimes customers demand complete isolation so they get their own cluster (although somehow its ok that the nodes are still VMs that are probably running on shared nodes… these requirements can be arbitrary sometimes).

> It helps to sometimes keep stateful (databases generally) workloads on one cluster, have another for stateless workloads etc. Sometimes customers demand complete isolation

Are these not covered by taint/toleration? I guess maybe isolation depending on what exactly they're demanding but even then I'd think it could work.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#30
post #17

Oh boy. Please, please stop using Medium for anything. I have lost count of how many potentially interesting or informative articles are published behind the Medium sign-in wall. At least for me, if you aren't publishing blog articles in public then what's the point of me trying to read them.

> "Create an account to read the full story." Why is this required?

Money
Post reply on HN