Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

blog.stackademic.com

Post reply on HN
Print view View original

211–216 of 216 posts

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#211

Post by bigfatkitten »

withinboredom wrote: post #174
Elidrake24 wrote: post #134

Earlier quoted context omitted.

Which in many cases would break SOC2 compliance (co-mingling of development and customer resources), and even goes against the basic advice offered in the K8s manual. Beyond that, this limits your ability to test Control Plane upgrades against your stack, though that has generally been very stable in my experience. To be clear I'm not defending the 47 Cluster setup of the OP, just the practice of separating Developme…

Why would you commingle development and customer resources? A k8s cluster is just a control plane, that specifically controls where things are running, and if you specify they can’t share resources, that’s the end of that. If you say they share the same control plane is commingling… then what do you think a cloud console is? And if you are using different accounts there… then I hope you are using dedicated resources…

> And if you are using different accounts there

Which for separating dev and prod, you absolutely should be.

(Separate accounts for AWS; separate projects would suffice for GCP.)

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#212

Post by bigfatkitten »

withinboredom wrote: post #199
_hl_ wrote: post #198

Earlier quoted context omitted.

The frustrating thing with SOC2, or pretty much most compliance requirements, is that they are less about what’s “technically true”, and more about minimizing raised eyebrows. It does make some sense though. People are not perfect, especially in large organizations, so there is value in just following the masses rather than doing everything your own way.

Yes. But it also isn’t a regulation. It is pretty much whatever you say it is.

The problem is you need to be able to convince the auditor that your controls meet the requirement. That's a much easier discussion to have with robust logical or physical separation.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#213

Post by bigfatkitten »

Zambyte wrote: post #153
nwatson wrote: post #133

Earlier quoted context omitted.

The constraint often would be regulatory. Even if technically isolation is possible, management won't risk SOC2 or GDPR non-compliance.

SOC2 is voluntary, not regulatory.

It's not voluntary if your customers have signed contracts with you on the basis that you gain and maintain that certification. And if they haven't, you shouldn't have wasted your money.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#214

Post by pm90 »

oofbey wrote: post #28
pm90 wrote: post #27

Earlier quoted context omitted.

There are genuine reasons for running multiple clusters. It helps to sometimes keep stateful (databases generally) workloads on one cluster, have another for stateless workloads etc. Sometimes customers demand complete isolation so they get their own cluster (although somehow its ok that the nodes are still VMs that are probably running on shared nodes… these requirements can be arbitrary sometimes).

Honest question - why would you want stateful workloads in a separate cluster from stateless? Why not just use a namespace?

because namespaces aren’t a failure boundary.

If your api gets hosed, you can create a new cluster and tear down the old one and call it a day.

With a stateful cluster, you can’t do that. As such you put in a lot more care with eg k8s upgrades, or introducing new controllers or admissions/mutating webhooks.

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#215

Post by pm90 »

yjftsjthsd-h wrote: post #29
pm90 wrote: post #27

Earlier quoted context omitted.

There are genuine reasons for running multiple clusters. It helps to sometimes keep stateful (databases generally) workloads on one cluster, have another for stateless workloads etc. Sometimes customers demand complete isolation so they get their own cluster (although somehow its ok that the nodes are still VMs that are probably running on shared nodes… these requirements can be arbitrary sometimes).

> It helps to sometimes keep stateful (databases generally) workloads on one cluster, have another for stateless workloads etc. Sometimes customers demand complete isolation Are these not covered by taint/toleration? I guess maybe isolation depending on what exactly they're demanding but even then I'd think it could work.

Yes only at the node (data plane) level but not the api (control plane).

Re: I Stopped Using Kubernetes. Our DevOps Team Is Happier Than Ever

#216

Post by pm90 »

wasmitnetzen wrote: post #84
pm90 wrote: post #27

Earlier quoted context omitted.

There are genuine reasons for running multiple clusters. It helps to sometimes keep stateful (databases generally) workloads on one cluster, have another for stateless workloads etc. Sometimes customers demand complete isolation so they get their own cluster (although somehow its ok that the nodes are still VMs that are probably running on shared nodes… these requirements can be arbitrary sometimes).

Yes, a few, maybe even 10, 12, but 47? It's also a prime number, so it's not something like running each thing three times for dev, stage and prod.

yeahhhh 47 seems insane
Post reply on HN