Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is your Kubernetes nightmare?

news.ycombinator.com

Post reply on HN
Print view View original

161–170 of 263 posts

Re: Ask HN: What is your Kubernetes nightmare?

#161

Post by yonixw »

_8j50 wrote: post #143
ekidd wrote: post #54

It's odd, but I actually really enjoy using Kubernetes in production. We have a few rules: 1. Read a good intro book cover-to-cover before trying to understand it. 2. Pay a cloud vendor to supply a working, managed Kubernetes cluster. 3. Prefer fewer larger clusters with namespaces (and node pools if needed) to lots of tiny clusters. 3. Don't get clever with Kubernetes networking. In fact, touch it as little as possi…

And don't expose workloads to the internet unless it is a prod app. Can you recommend me a good intro book to read cover to cover (hopefully not too thick).

"Kubernetes in Action" by Marko (Manning publishing) is my recommendation. Took me from someone who knows docker/docker-compose to someone who can handle Azure/AWS Kubernetes, understand the terms and design apps. Very good book.

Re: Ask HN: What is your Kubernetes nightmare?

#162

Post by wg0 »

ekidd wrote: post #54

It's odd, but I actually really enjoy using Kubernetes in production. We have a few rules: 1. Read a good intro book cover-to-cover before trying to understand it. 2. Pay a cloud vendor to supply a working, managed Kubernetes cluster. 3. Prefer fewer larger clusters with namespaces (and node pools if needed) to lots of tiny clusters. 3. Don't get clever with Kubernetes networking. In fact, touch it as little as possi…

Good rules.

>2. Pay a cloud vendor to supply a working, managed Kubernetes cluster.

If one is at that level already, I don't think there's anything better than AWS ECS out there. It just works. Just works. Yes sure, it does not offer stateless workloads for example among other things but it works for 90% of the cases.

> 3. Don't get clever with Kubernetes networking. In fact, touch it as little as possible and hope really hard it continues to work.

Pretty much... Each CNI generates the SDN its own way slightly differing then the others. It is like you can write the program to print a chessboard on terminal in ten different ways.

Unfortunately, these implementation details aren't written or documented anywhere and they of course would keep changing from release to release anyway. Your only way out if you have production workloads that you can't afford going down without missing revenue? Just pay for the support for respective CNI as only they would know what the voodoo magic is under the hood.

Sure you can see the source code and all of them are open source but that's not your main business or the main day job and of course, the solutions aren't 100 line trivial implementations either.

Re: Ask HN: What is your Kubernetes nightmare?

#163

Post by yonixw »

sateesh wrote: post #92
jimbokun wrote: post #64

Earlier quoted context omitted.

What are some of the best Kubernetes books?

Kubernetes in Action by Manning ( https://www.manning.com/books/kubernetes-in-action ) is quite through, good and beginner friendly.

Second that, But I do recommend to come with docker/docker-compose understanding in advance.

Re: Ask HN: What is your Kubernetes nightmare?

#164

Post by mrweasel »

treis wrote: post #156
ekidd wrote: post #54

It's odd, but I actually really enjoy using Kubernetes in production. We have a few rules: 1. Read a good intro book cover-to-cover before trying to understand it. 2. Pay a cloud vendor to supply a working, managed Kubernetes cluster. 3. Prefer fewer larger clusters with namespaces (and node pools if needed) to lots of tiny clusters. 3. Don't get clever with Kubernetes networking. In fact, touch it as little as possi…

>Prefer fewer larger clusters with namespaces (and node pools if needed) to lots of tiny clusters. People do this? I thought the whole point was to abstract everything away. You should have containers running on pods. You shouldn't care about what's in the containers or what metal the pods are running on.

Some people don't trust the namespacing in Kubernetes, or have contractual obligations to keep environments separate. I've rarely seen clusters with more than 10 nodes, but I have seen single customers run 5 tiny different clusters, for different environment.

Re: Ask HN: What is your Kubernetes nightmare?

#165

Post by anderiv »

wg0 wrote: post #162
ekidd wrote: post #54

It's odd, but I actually really enjoy using Kubernetes in production. We have a few rules: 1. Read a good intro book cover-to-cover before trying to understand it. 2. Pay a cloud vendor to supply a working, managed Kubernetes cluster. 3. Prefer fewer larger clusters with namespaces (and node pools if needed) to lots of tiny clusters. 3. Don't get clever with Kubernetes networking. In fact, touch it as little as possi…

Good rules. >2. Pay a cloud vendor to supply a working, managed Kubernetes cluster. If one is at that level already, I don't think there's anything better than AWS ECS out there. It just works. Just works. Yes sure, it does not offer stateless workloads for example among other things but it works for 90% of the cases. > 3. Don't get clever with Kubernetes networking. In fact, touch it as little as possible and hope r…

> If one is at that level already, I don't think there's anything better than AWS ECS out there.

100%. To answer the OP's question: my nightmare is having to use it at all. I work with small, very early-stage companies whose applications by and large are not complicated. Perhaps at some level of scale and/or complexity, k8s makes sense. For the vast majority of the cases I see, something like ECS does everything they need, while being significantly more simple to understand, develop for, and debug.

Re: Ask HN: What is your Kubernetes nightmare?

#166

Post by cortesoft »

SoftTalker wrote: post #118
Intermernet wrote: post #66

Earlier quoted context omitted.

Kubernetes on bare metal is actually pretty easy. Kubernetes on a hosted solution which doesn't have a managed version is prone to error. Usually on bare metal you can make some guarantees regarding bandwidth and storage speed. Trying to roll out a cluster on a service that can't give you these guarantees is truly a nightmare.

I have tried several times over the past few years to install Kubernetes on bare metal, and it has never worked. I don't mean installing it on VMs on a laptop, I mean on a real linux cluster of 8 to 32 nodes, with real networks and real switches. Managing bare metal machines is a cakewalk compared to getting Kubernetes running in-house, at least in my experience. Obviously the cloud providers do it, so it's possible.…

What were you using to install kubernetes?

Re: Ask HN: What is your Kubernetes nightmare?

#167

Post by pojzon »

ekidd wrote: post #54

It's odd, but I actually really enjoy using Kubernetes in production. We have a few rules: 1. Read a good intro book cover-to-cover before trying to understand it. 2. Pay a cloud vendor to supply a working, managed Kubernetes cluster. 3. Prefer fewer larger clusters with namespaces (and node pools if needed) to lots of tiny clusters. 3. Don't get clever with Kubernetes networking. In fact, touch it as little as possi…

> 3. Prefer fewer larger clusters with namespaces (and node pools if needed) to lots of tiny clusters.

This is interesting - last time I worked with Microsoft Engineers from Azure - they said exactly the opposite.

One workload = One cluster.

„There are too many shared resources in Kubernetes that can leak collateral damage from one workload to another”.

Re: Ask HN: What is your Kubernetes nightmare?

#168

Post by tazjin »

fn1 wrote: post #116
tazjin wrote: post #105

Earlier quoted context omitted.

As a Xoogler I'd say that Kubernetes is harder to use than Google's internal equivalents. It may not be harder to run, but that doesn't matter inside of Google unless you're on the teams responsible for the base layers. My point is, Kubernetes isn't really "made for Google level complexity" - Google only uses it for a handful of cloud products, internal research stuff and not much else.

Why did Google then release the complex Kubernetes to the world than their simpler internal tools?

It's basically impossible to release this kind of internal tooling due to the way Google works. You'd have to rewrite it, or open-source essentially the entire foundation, and Google probably doesn't want to do either.

Re: Ask HN: What is your Kubernetes nightmare?

#169

Post by wg0 »

vbezhenar wrote: post #159

Some very basic things look very hard to me. Right now I’m scratching my head how do I implement even non-HA WireGuard server in a pod, so wg clients can access a pod network and pods can access wg client network. Seems like a very basic requirement for any installation yet zero guides about it. And don’t even talk about HA server with load balancer.

If you find anything about it, please do share. I recall I saw something similar somewhere. Maybe it was OpenVPN.

Re: Ask HN: What is your Kubernetes nightmare?

#170

Post by dewey »

pojzon wrote: post #167
ekidd wrote: post #54

It's odd, but I actually really enjoy using Kubernetes in production. We have a few rules: 1. Read a good intro book cover-to-cover before trying to understand it. 2. Pay a cloud vendor to supply a working, managed Kubernetes cluster. 3. Prefer fewer larger clusters with namespaces (and node pools if needed) to lots of tiny clusters. 3. Don't get clever with Kubernetes networking. In fact, touch it as little as possi…

> 3. Prefer fewer larger clusters with namespaces (and node pools if needed) to lots of tiny clusters. This is interesting - last time I worked with Microsoft Engineers from Azure - they said exactly the opposite. One workload = One cluster. „There are too many shared resources in Kubernetes that can leak collateral damage from one workload to another”.

Wouldn't a name space make more sense than a whole cluster?
Post reply on HN