Live data from Hacker News

Ask HN: Do's/don'ts of working with Kubernetes you learned through experience?

news.ycombinator.com

21–30 of 30 posts

Re: Ask HN: Do's/don'ts of working with Kubernetes you learned through experience?

#21

Can I be honest? As someone who hopped on the K8s bandwagon back in the early days (circa-early 2017), _do not_ go into production with Kubernetes if you're still asking this question. Just a few of the issues I've run into over the past 2 1/2 years or so: - Kubernetes DNS flaking out completely - Kubernetes DNS flaking out occasionally (for ~5 percent of queries) - Giving out too many permissions, causing pods to be…

What about going into prod with a managed K8s? EKS or GKE?

Re: Ask HN: Do's/don'ts of working with Kubernetes you learned through experience?

#22

We're moving away from Kubernetes, to Aptible. If you're asking these kinds of questions you shouldn't be using kubernetes. If you are going to use it, be ready to have an engineer on your time be full time devops. Or be ready to hire someone who knows k8. It'll be around 110k to 140k. But really, don't use it. The gospel you hear is from engineers who already invested their careers in it. Buyer beware.

Something I'm struggling to understand about the hype of orchestrating a container architecture for everything you do is, if it's so incredibly nuanced, full of pitfalls, and k8 takes 80 hours of instruction/practice to full grasp how "easy" it makes life (despite there being layers on top of it like openshift that most orgs are gonna use anyway) and takes at least one "devops" person on a six figure salary to make sure it all doesnt crash and burn.. what was the point?

Re: Ask HN: Do's/don'ts of working with Kubernetes you learned through experience?

#23
post #2

See the excellent collection of Kubernetes incident reviews at https://k8s.af

Wow, there are at least two linked stories where people are running pods without resource limit. Without. Resource. Limit.

At my workplace, I've been whining that we really need resource limit for... well much longer than I'd like... while thinking few other places will be crazy enough to think you can just start jobs without limits. But apparently many people do! Why are we running Kubernetes, again?

Re: Ask HN: Do's/don'ts of working with Kubernetes you learned through experience?

#24
post #22

We're moving away from Kubernetes, to Aptible. If you're asking these kinds of questions you shouldn't be using kubernetes. If you are going to use it, be ready to have an engineer on your time be full time devops. Or be ready to hire someone who knows k8. It'll be around 110k to 140k. But really, don't use it. The gospel you hear is from engineers who already invested their careers in it. Buyer beware.

Something I'm struggling to understand about the hype of orchestrating a container architecture for everything you do is, if it's so incredibly nuanced, full of pitfalls, and k8 takes 80 hours of instruction/practice to full grasp how "easy" it makes life (despite there being layers on top of it like openshift that most orgs are gonna use anyway) and takes at least one "devops" person on a six figure salary to make s…

After a few weeks of diving into it, the point is that you can scale from tiny 1 pod, to huge enterprise Google-scale infrastructure.

But realistically, you don't need all that bullshit when you're a startup or even a large company making 7 figures.

Re: Ask HN: Do's/don'ts of working with Kubernetes you learned through experience?

#26

Can I be honest? As someone who hopped on the K8s bandwagon back in the early days (circa-early 2017), _do not_ go into production with Kubernetes if you're still asking this question. Just a few of the issues I've run into over the past 2 1/2 years or so: - Kubernetes DNS flaking out completely - Kubernetes DNS flaking out occasionally (for ~5 percent of queries) - Giving out too many permissions, causing pods to be…

Thanks for describing your experience.

You do suggest not going into production with k8s, but what if the alternative is as difficult to implement or worse?

Re: Ask HN: Do's/don'ts of working with Kubernetes you learned through experience?

#28
post #2

See the excellent collection of Kubernetes incident reviews at https://k8s.af

Thanks for mentioning my collection of Kubernetes Failure Stories! I'll use this opportunity to encourage everyone to submit their own postmortems :-) --> https://github.com/hjacobs/kubernetes-failure-stories

Re: Ask HN: Do's/don'ts of working with Kubernetes you learned through experience?

#29
Have a very clear business case of why you are using it.

Rolling out K8s should not be the goal. It’s a toolset, an expensive a bleeding edge one. It’s also very much geared for operators not developers so you likely need to build guide rails on top of it.

There are lots of good reasons to use K8s but make sure you know why you are.

Re: Ask HN: Do's/don'ts of working with Kubernetes you learned through experience?

#30

If your application requires high availability, make sure you are setting pod disruption budgets and have some special behavior when SIGKILL is sent to an app/pod. For some of our applications, we have some logic to finish all current requests after SIGKILL is sent, so that none are dropped.

You mean SIGTERM, right? That's what gets sent to the containers before the grace period expires.

Yes, my mistake.
Post reply on HN