Live data from Hacker News

Kubernetes Failure Stories

srcco.de

61–70 of 242 posts

Re: Kubernetes Failure Stories

#61
post #51

It's not for everyone and it has significant maintenance overhead if you want to keep it up to date _and_ can't re-create the cluster with a new version every time. This is something most people at Google are completely insulated from in the case of Borg, because SRE's make infrastructure "just work". I wish there was something drastically simpler. I don't need three dozen persistent volume providers, or the ability…

I'm not sure but would something like docker swarm qualify?

Re: Kubernetes Failure Stories

#62
post #3

I'd be interested in a related "microservices failure stories". Must be a big overlap with this.

I have two. One was caused by data inconsistency between services and regions. One is more hypothetical: the microservices had gotten to the point that no one knew how to start the system if all services are down, and it's possible that services have circular dependencies to the point that it would be incredibly hard to do a cold start.

You handle the hypothetical by validating a new environment can be built and bootstrapped. Doing this on a regular basis, either by tearing down and rebuilding dev, or in a separate environment just for this purpose, is not a "nice to have". This same problem exists with monoliths with complicated dependencies, nothing new here.

Just like backups, if infrastructure as code isn't tested, it's worthless.

Re: Kubernetes Failure Stories

#63
post #50

Kubernetes solves a problem that most of the companies don't have. That is why I don't understand why the hype around it is so big. For the majority, it just adds a little value when you compare to added complexity to infrastructure and the cost of a learning curve and the ongoing operation and maintenance.

In my experience most companies lack common conventions and automations.

Kubernetes "done right" is almost a part of your application. It becomes this "machine" that you throw stuff into and good stuff happens.

You'll need a team to integrate it into the pieces you require (auth, secrets, loadbalancers, permissions/app identities, monitoring and logging) but many places lack bits and pieces, and in my opinion k8s gives you a fast track to create a uniform application delivery platform.

What I don't like is that it kind of is the opposite of "the unix philosophy" and in that regard I prefer the hashicorp stack.

Re: Kubernetes Failure Stories

#64
post #51

It's not for everyone and it has significant maintenance overhead if you want to keep it up to date _and_ can't re-create the cluster with a new version every time. This is something most people at Google are completely insulated from in the case of Borg, because SRE's make infrastructure "just work". I wish there was something drastically simpler. I don't need three dozen persistent volume providers, or the ability…

An open source having complexity of an enterprise monster - this is what generates the half million plus salaries. An old enterprise software trick. Simplification of it would serve no interests of anybody in the position to do the simplification.

Re: Kubernetes Failure Stories

#65

Having used Docker Compose/Swarm for last two years, I remember having problems with them twice. One of which was an MTU setting which I didn't really understand why, but overall I was relatively happy with them. Since Kubernetes seems to have won, I decided to learn it but got some disappointments. The first disappointment is setting up a local development environment. I failed to get minikube running on a Macbook A…

Or you could just learn how to manage infrastructure the old fashioned way, which was never broken for small business and mid-sized enterprise environments. The only time you need the complexity and overhead of something like kubernetes is when you are truly large or when you have caught the in-fashion disease.

It's quite simple for a 20 year SA to stand up a highly integrated environment with modular monitoring, directory service, virtualization and hybrid cloud options for all services in a week. Why don't you hire one of these for the job instead of recipe/containering yourself into 'doesn't work, I dunno' posts.

Re: Kubernetes Failure Stories

#66
post #51

It's not for everyone and it has significant maintenance overhead if you want to keep it up to date _and_ can't re-create the cluster with a new version every time. This is something most people at Google are completely insulated from in the case of Borg, because SRE's make infrastructure "just work". I wish there was something drastically simpler. I don't need three dozen persistent volume providers, or the ability…

> leaving no option but to move somewhere else

Many of the major infrastructure/platform vendors are rolling out their own distribution of Kubernetes either as a cloud service e.g AWS, Azure, GCP or on premise e.g. RedHat.

So I suspect they are going to try and differentiate on features and ease of use and make it as hard as possible to move anywhere else.

Re: Kubernetes Failure Stories

#67
post #50

Kubernetes solves a problem that most of the companies don't have. That is why I don't understand why the hype around it is so big. For the majority, it just adds a little value when you compare to added complexity to infrastructure and the cost of a learning curve and the ongoing operation and maintenance.

>> I don't understand why the hype around it is so big

Probably because it started at Google, if it was created by IBM then we'd only hear about it on TV ads.

Re: Kubernetes Failure Stories

#68

Earlier quoted context omitted.

I've had small-ish docker swarms in production for a couple of years as well, and I really don't understand why it doesn't seem to be popular at all. I feel like I need to move to K8S just because swarm seems to be going away, but I'm really not seeing the technical advantages at all. If someone could point me to an article explaining why k8s is so much better than swarm, I'd really appreciate it. Are the big advanta…

I'm also constantly surprised at how unpopular docker swarm is given that everyone already uses docker itself. Why do you think swarm is going away though? I love the idea of just using my docker compose file as my deployment config.

I'm not familiar with Docker Swarm, but the Kubernetes API is the major strong point for me (not saying that it's perfect): it has the right abstractions (CronJob, StatefulSet, ..) and is extensible (Custom Resource Definitions). There are many ways to run containerized workloads (ECS, Mesos, Docker Swarm, ..), but the de-facto agreement on the Kubernetes API is a game changer: now we can start building things on top of it :-)

Re: Kubernetes Failure Stories

#69
post #51

It's not for everyone and it has significant maintenance overhead if you want to keep it up to date _and_ can't re-create the cluster with a new version every time. This is something most people at Google are completely insulated from in the case of Borg, because SRE's make infrastructure "just work". I wish there was something drastically simpler. I don't need three dozen persistent volume providers, or the ability…

Re configuration: ksonnet is an option (although I personally find jsonnet a “lipstick on a pig” kind of solution).

There’s some work going on to have something more user-friendly (think Google’s Piccolo) - https://github.com/stripe/skycfg (disclaimer - I contributed to this project)

Re: Kubernetes Failure Stories

#70
post #50

Kubernetes solves a problem that most of the companies don't have. That is why I don't understand why the hype around it is so big. For the majority, it just adds a little value when you compare to added complexity to infrastructure and the cost of a learning curve and the ongoing operation and maintenance.

> Kubernetes solves a problem that most of the companies don't have

Actually most medium to large companies do have this problem.

There are often a lot of different languages, libraries, versions, deployment methods etc. And the appeal of Docker was that you can treat them all as block boxes. And the appeal of Kubernetes is that you have this rich support infrastructure to run them all hands-off at scale.

It definitely solves a problem. Just not particularly well.

Post reply on HN