Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is your Kubernetes nightmare?

news.ycombinator.com

Post reply on HN
Print view View original

101–110 of 263 posts

Re: Ask HN: What is your Kubernetes nightmare?

#101

Post by sleepybrett »

Intermernet wrote: post #66
ekidd wrote: post #54

It's odd, but I actually really enjoy using Kubernetes in production. We have a few rules: 1. Read a good intro book cover-to-cover before trying to understand it. 2. Pay a cloud vendor to supply a working, managed Kubernetes cluster. 3. Prefer fewer larger clusters with namespaces (and node pools if needed) to lots of tiny clusters. 3. Don't get clever with Kubernetes networking. In fact, touch it as little as possi…

Kubernetes on bare metal is actually pretty easy. Kubernetes on a hosted solution which doesn't have a managed version is prone to error. Usually on bare metal you can make some guarantees regarding bandwidth and storage speed. Trying to roll out a cluster on a service that can't give you these guarantees is truly a nightmare.

I would also say that if you are going to be administering clusters at your company that you should at least set up a cluster from scratch (doesn't have to be bare metal) and learn how the kubernetes control plane works by breaking it in various ways etc.

In my experience most people don't like black magic, they want something that they understand on some level. A fully managed k8s cluster is black magic, once you have set up a vanilla cluster you get a much better feeling about how the control plane works together to get things done.

Re: Ask HN: What is your Kubernetes nightmare?

#102

Post by sleepybrett »

spicyusername wrote: post #34

- 60% of the Kubernetes ecosystem is half-baked alpha software - Maintaining 200+ clusters for 10 small applications - Cloud bills - Autoscaling never working well - Trying to untangle Terraform state without taking down Prod

Tell me you barely understand kubernetes without saying you barely understand kuberentes.

Re: Ask HN: What is your Kubernetes nightmare?

#104

Post by roflyear »

cyberpunk wrote: post #49
roflyear wrote: post #38

Earlier quoted context omitted.

Never in my experience when someone says it just works does it just work in reality. Usually they are hiding some things that they are keeping up with but for some reason don't include those issues as the thing not working.

I look after a lot of k8s professionally (hundreds of clusters), and the only issues I've had in the last few years were a istio/pilotd bug and a bit of pain moving from e.g v1beta1->v1 during normal upgrades. Everything else that's gone wrong with these environments was cloud/hw related, or app stuff. Guess it depends on where your competencies are.

> Guess it depends on where your competencies are.

That's true of almost everything, no?

I use k8s myself (nowhere near that scale, basically the smallest scale you can think of... one cluster...) and don't think it is a bad product.

Re: Ask HN: What is your Kubernetes nightmare?

#105

Post by tazjin »

oxff wrote: post #45

You aren't supposed to use a tool made for Google level complexity unless you work with such complexity in the first place.

As a Xoogler I'd say that Kubernetes is harder to use than Google's internal equivalents. It may not be harder to run, but that doesn't matter inside of Google unless you're on the teams responsible for the base layers.

My point is, Kubernetes isn't really "made for Google level complexity" - Google only uses it for a handful of cloud products, internal research stuff and not much else.

Re: Ask HN: What is your Kubernetes nightmare?

#106

Post by seized »

ColdHeat wrote: post #100
garciasn wrote: post #76

Earlier quoted context omitted.

We use GKE. 1. I don't know about any of this; we don't seem to have problems. 2. This sounds like an architecture issue, not a k8s issue. 3. Our entire GKE infrastructure costs less than $50 a month. 4. You're right here; it doesn't work 'well', but it works 'well enough' for our use cases. 5. I'm sure you're talking about some event that was far more complex than the few times we've had to drain our pool, but we di…

With regards to 3, isn’t the management plane alone $72 for GKE without considering the cost for the nodes? How are your costs so low?

GKE gives you one free cluster.

https://cloud.google.com/kubernetes-engine/pricing

Re: Ask HN: What is your Kubernetes nightmare?

#107

Post by drbojingle »

guywhocodes wrote: post #93
oxff wrote: post #45

You aren't supposed to use a tool made for Google level complexity unless you work with such complexity in the first place.

Where do I go for my 3-5 node cluster which I want to schedule containers to, run dynamic workloads etc? How do I build a container based DAG in a small cluster today without k8s? Solutions that are not k8s tend to be single programming language/sdk based or not easier to set up. I get the feeling the alternatives are dying off unnaturally fast.

What exactly are you making that requires the use of multiple languages?

Re: Ask HN: What is your Kubernetes nightmare?

#108

Post by xiwenc »

tpxl wrote: post #97
phaer wrote: post #89

Earlier quoted context omitted.

Why would you need a load-balancer if you only have a single machine?

Because kubernetes says so? Can you run it without a load balancer?

You can use Nodeport instead of loadbalancer. Or use metallb if you insist to have LB so that it’s more closer to real production environments.

Re: Ask HN: What is your Kubernetes nightmare?

#109

Post by spicyusername »

sleepybrett wrote: post #102
spicyusername wrote: post #34

- 60% of the Kubernetes ecosystem is half-baked alpha software - Maintaining 200+ clusters for 10 small applications - Cloud bills - Autoscaling never working well - Trying to untangle Terraform state without taking down Prod

Tell me you barely understand kubernetes without saying you barely understand kuberentes.

Well, that is a rude thing to say.
Post reply on HN