Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is your Kubernetes nightmare?

news.ycombinator.com

Post reply on HN
Print view View original

191–200 of 263 posts

Re: Ask HN: What is your Kubernetes nightmare?

#191

Post by mschuster91 »

MuffinFlavored wrote: post #189
bradwood wrote: post #184

Earlier quoted context omitted.

I've done exactly that. It wasn't fun. RHEL, k3s, ansible, Longhorn, metallb, to name a few. Storage is the fun part.

I was playing around with a local Elasticsearch cluster and I couldn't figure out how to "do" k8s storage. Some kind of like... shared NFS volume or something maybe? Could you spread some tips from your experiences?

Yeah, NFS is your best bet if you are on a lab enviroment. Mount the volume at each host, then use hostPath to mount it in the pods (https://kubernetes.io/docs/concepts/storage/volumes/#hostpat...).

Re: Ask HN: What is your Kubernetes nightmare?

#192

Post by legohead »

I don't like to upgrade things that are working. Unfortunately that bit me in the butt with Kubernetes.

Long story short, a node crashed, and when it came back up, the pods wouldn't start. We spent a couple days trying to figure it out, but nothing was working. This was in production, so we made the choice to rebuild the entire cluster again with a newer version. We still had other nodes running, and were scaled enough that there was no complete downtime, but we were maxxing the cpu and some connections were getting dropped.

Re: Ask HN: What is your Kubernetes nightmare?

#193

Post by worried4future »

cncfNormalLady wrote: post #172

My Kubernetes nightmare is not the software but the strident, intolerant, social justice keyboard warriors involved with the CNCF and k8s technologies. They make up a tiny minority but they are loud and often nasty within their Twitter echo chamber and they are excellent at getting companies and people to do their bidding out of fear of reprisal. It’s supposed to be the “most welcoming community” but it only takes yo…

It's a cult.

While I agree with you I don't think it's particularly worse in k8/CNCF than in other parts of tech.

Re: Ask HN: What is your Kubernetes nightmare?

#194

Post by fireflash38 »

pojzon wrote: post #167
ekidd wrote: post #54

It's odd, but I actually really enjoy using Kubernetes in production. We have a few rules: 1. Read a good intro book cover-to-cover before trying to understand it. 2. Pay a cloud vendor to supply a working, managed Kubernetes cluster. 3. Prefer fewer larger clusters with namespaces (and node pools if needed) to lots of tiny clusters. 3. Don't get clever with Kubernetes networking. In fact, touch it as little as possi…

> 3. Prefer fewer larger clusters with namespaces (and node pools if needed) to lots of tiny clusters. This is interesting - last time I worked with Microsoft Engineers from Azure - they said exactly the opposite. One workload = One cluster. „There are too many shared resources in Kubernetes that can leak collateral damage from one workload to another”.

It's really quite frustrating to find that out from experience. Things like operators, anything Cluster*, named resources, etc.

If you're doing relatively simple things with the cluster, then you can do namespaces. The more custom shit you do, the better off you are with true isolation.

Re: Ask HN: What is your Kubernetes nightmare?

#195

Post by fireflash38 »

MuffinFlavored wrote: post #189
bradwood wrote: post #184

Earlier quoted context omitted.

I've done exactly that. It wasn't fun. RHEL, k3s, ansible, Longhorn, metallb, to name a few. Storage is the fun part.

I was playing around with a local Elasticsearch cluster and I couldn't figure out how to "do" k8s storage. Some kind of like... shared NFS volume or something maybe? Could you spread some tips from your experiences?

Longhorn is honestly pretty easy to get up & going for backing a PersistentVolume, which you can then mount however you want.

K3S has some local storage options too, but that's of mixed usage. Or you just do a hostpath + NFS if you want something that has as little Kubernetes magic as possible.

Re: Ask HN: What is your Kubernetes nightmare?

#197

Post by icedchai »

dewey wrote: post #173
Intermernet wrote: post #66

Earlier quoted context omitted.

Kubernetes on bare metal is actually pretty easy. Kubernetes on a hosted solution which doesn't have a managed version is prone to error. Usually on bare metal you can make some guarantees regarding bandwidth and storage speed. Trying to roll out a cluster on a service that can't give you these guarantees is truly a nightmare.

> Kubernetes on bare metal is actually pretty easy. I would not call it easy at all. Last time I tried that a year ago you still needed a special load balancer to get it going ( https://metallb.universe.tf ). Has this changed?

MetalLB is pretty simply to configure.

Re: Ask HN: What is your Kubernetes nightmare?

#198

Post by cubancigar11 »

ekidd wrote: post #54

It's odd, but I actually really enjoy using Kubernetes in production. We have a few rules: 1. Read a good intro book cover-to-cover before trying to understand it. 2. Pay a cloud vendor to supply a working, managed Kubernetes cluster. 3. Prefer fewer larger clusters with namespaces (and node pools if needed) to lots of tiny clusters. 3. Don't get clever with Kubernetes networking. In fact, touch it as little as possi…

Which book would you suggest please?

Re: Ask HN: What is your Kubernetes nightmare?

#199

Post by drbojingle »

discreteevent wrote: post #124
drbojingle wrote: post #107

Earlier quoted context omitted.

What exactly are you making that requires the use of multiple languages?

It would seem that the requirement for multiple languages often has nothing to do with making something or the customer requirements. It's often a developer requirement. Developers want to work in their favorite language. In the 90s you just wrote in whatever the company or ecosystem had mandated. Now that developers are in high demand they can specify what they want to use. It started with books like "Beyond java" (…

Kinda what I expect as well. I could understand it if a company was bringing in whole teams to scale up development of a large project quickly. Even then support would could be a nightmare. Doing it just because sounds painful.

Re: Ask HN: What is your Kubernetes nightmare?

#200

Post by message »

CSDude wrote: post #17

I have 2: 1. We build our own custom build system, because there is no CI that can do actual DAGs (maybe a few). A custom Kubernetes operator that parses Jsonnet files to create 100s of CRDs and pods to achieve extreme parallelization. EKS was 144$/mo (now 72$) but no info on master node types. Using watch endpoints with hundreds of pods did not scale well. They had to bump up the master node instances to c5.18xlarge…

D'ya like dags?
Post reply on HN