Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is your Kubernetes nightmare?

news.ycombinator.com

Post reply on HN
Print view View original

21–30 of 263 posts

Re: Ask HN: What is your Kubernetes nightmare?

#21

Post by ygouzerh »

More distributed system than Kubernetes, but quite fun : We deployed a MongoDB cluster on our Kubernetes Clusters. Our application was a having a chat feature that stored the messages into the MongoDB cluster. After some months, we realized that we got some weird issues, some messages was arriving in the wrong orders, like : 1. A : Hi ! 2. B : Bye ! See you next time ! 3. A : Great and you ? 4. B : Hello ! How are you ?

We thought it was an application issue, but it was that actually on the database side : the timestamp of each message was using the local time of the mongodb instance. And between different instances, the time was different. We realized that the Kubernetes Nodes had issues to connect to the NTP server, due to a rule in an random firewall.

When we fixed it, every other messages where in the good order

Re: Ask HN: What is your Kubernetes nightmare?

#22

Post by wg0 »

ciguy wrote: post #2

What are you fishing for exactly here? K8s has it's issues, but the way you have phrased the question is only going to get you biased answers.

I am a k8s user myself and I believe it is one of the most innovative projects out there. My intent here is to reach out to the wider community to share my fears/pain points and see if:

- Is there a knowledge gap that I'm having and I can work on myself to be better? - Are there any practices, techniques, tricks, SOPs, solutions, mechanisms that wider community has developed or still under evolution that I can put at use.

I know k8s has issues but knowing those issues up close (including why they are) can make one a better operator of the machinery at hand.

So mainly, I'm seeking advice, opinions, views that come from people's experiences and aren't necessarily in official docs or books/courses/workshops/seminars etc.

Re: Ask HN: What is your Kubernetes nightmare?

#23

Post by jokethrowaway »

Honestly I don't get the hate k8s get.

I run my own clusters and it just works.

Sure I have to ignore a lot of crap in the setup phase, there are so many products out there I don't want to pay for. The nightmare may come from some devop installing a bunch of helm charts without configuring things properly.

Scaling down to a minimal cluster is a real concern: I would like to run k8s for some micro project that literally run on 5$ vps but it's too heavy for that.

Re: Ask HN: What is your Kubernetes nightmare?

#24

Post by sureglymop »

For me it is actually getting services exposed. Say i buy a domain and then i (trivially) containerize my applications and set up services in kubernetes. Now comes the networking part which is just a pain. How do I make my service accessible? It's easy with docker and an nginx reverse proxy but with kubernetes it's always seemed to be a real pain.

Re: Ask HN: What is your Kubernetes nightmare?

#25

Post by sureglymop »

melezhik wrote: post #14

This is that I have to use yaml to configure k8s. Every k8sish tooling has it's own yaml API, including helm, gitops, argocd and friends, so you end up having a bunch of brittle and very hard to understand and maintain yaml files ... Sigh

I think that it's just the "kubectl" tool which accepts yaml input. This tool then talks to the actual API of your cluster and uses json (and maybe other formats).

Re: Ask HN: What is your Kubernetes nightmare?

#27

Post by GordonS »

robertlagrant wrote: post #15
JensRantil wrote: post #4

Honestly, creating my first K8s deployment of a service; typing out at least 150 lines of YAML to define my Deployment, figuring out how my ConfigMaps, Secrets, and Volumes, Services are defined and connected together. Vanilla K8s YAML is extremely low-level.

Having experienced many Word documents full of deployment instructions and screenshots on how to deploy software, a few lines of YAML has been amazing for me :) But no doubt there are other tools that are even better.

Compare with Docker Compose files, which still use YAML, but a lot less of it, and less verbose too.

Re: Ask HN: What is your Kubernetes nightmare?

#28

Post by 7sidedmarble »

sureglymop wrote: post #24

For me it is actually getting services exposed. Say i buy a domain and then i (trivially) containerize my applications and set up services in kubernetes. Now comes the networking part which is just a pain. How do I make my service accessible? It's easy with docker and an nginx reverse proxy but with kubernetes it's always seemed to be a real pain.

Just making your pod accessible is not as complicated as it seems. All you need are: 1. A kubernetes service resource. This just contains a selector that points at your pod. 2. An ingress. You point this at the service you just made. You will get a static IP. Point your DNS at that.

And you're good to go. This assumes that you are using a provider that comes with an 'ingress controller' out of the box (which is what actually makes the ingress function. It's usually just nginx). If not, install the nginx ingress controller with helm. Then install cert-manager with helm for tls cert provisioning.

Re: Ask HN: What is your Kubernetes nightmare?

#29

Post by high_5 »

I'm afraid k8s would become like git - it's a great tool out of which we mostly use like 5% of it's complete capabilities, yet we all use it because everyone is using it. Yet, k8s doesn't really make all the underlying stacks go away. When the shit hits the fan, you have to troubleshoot it with knowledge of much more than just YAML syntax.

Re: Ask HN: What is your Kubernetes nightmare?

#30

Post by p_l »

JensRantil wrote: post #4

Honestly, creating my first K8s deployment of a service; typing out at least 150 lines of YAML to define my Deployment, figuring out how my ConfigMaps, Secrets, and Volumes, Services are defined and connected together. Vanilla K8s YAML is extremely low-level.

The most freeing moment with k8s for me was integrating that no, you don't need to write all that YAML (or JSON), and going into generating manifests from other formats - in my case, usually Jsonnet with mix of libraries both public and private which quickly embedded knowledge specific to our setup.

I think one of the best things I ever heard from another colleague was "I took the example jsonnet and had working version for completely new deployment within hour" :)

Post reply on HN