Live data from Hacker News

Kubernetes Failure Stories

srcco.de

51–60 of 242 posts

Re: Kubernetes Failure Stories

#51
It's not for everyone and it has significant maintenance overhead if you want to keep it up to date _and_ can't re-create the cluster with a new version every time. This is something most people at Google are completely insulated from in the case of Borg, because SRE's make infrastructure "just work". I wish there was something drastically simpler. I don't need three dozen persistent volume providers, or the ability to e.g. replace my network plugin or DNS provider, or load balancer. I want a sane set of defaults built-in. I want easy access to persistent data (currently a bit of a nightmare to set up in your own cluster). I want a configuration setup that can take command line params without futzing with templating and the like. As horrible and inconsistent as Borg's BCL is, it's, IMO, an improvement over what K8S uses.

Most importantly: I want a lot fewer moving parts than it currently has. Being "extensible" is a noble goal, but at some point cognitive overhead begins to dominate. Learn to say "no" to good ideas.

Unfortunately there's a lot of K8S configs and specific software already written, so people are unlikely to switch to something more manageable. Fortunately if complexity continues to proliferate, it may collapse under its own weight, leaving no option but to move somewhere else.

Re: Kubernetes Failure Stories

#52
post #43

Having used Docker Compose/Swarm for last two years, I remember having problems with them twice. One of which was an MTU setting which I didn't really understand why, but overall I was relatively happy with them. Since Kubernetes seems to have won, I decided to learn it but got some disappointments. The first disappointment is setting up a local development environment. I failed to get minikube running on a Macbook A…

Digital Ocean's K8S offering is out of beta now: https://www.digitalocean.com/products/kubernetes/

It’s good, but the storage layer has some bugs. For example, if you create a pvc, then resize the volume according to their docs, the new size doesn’t reflect in k8s. Also you can create pv’s manually and it won’t show up in the dashboard.

Re: Kubernetes Failure Stories

#53
post #49

I am a developer and I find k8s frustrating. To me, its documentation is confusing and scattered among too many places (best example: overlay networks). I have read multiple books and gazillions of articles and yet I have the feeling that I am lacking the bigger picture. I was able to set it up successfully a couple of times, with more or less time required. Last time, I gave up after four days because I realized tha…

Have you used other google products? I find their documentation routinely incomprehensible and difficult.

Re: Kubernetes Failure Stories

#54

I've started the planning phase of a Kubernetes course, geared toward developers more so than the enterprise gatekeepers. As I read stories like these, I jump between different thoughts and feelings: 1) no matter what I think I know, there's too many dark corners to create an adequate course 2) K8S is such a dumpster fire that I shouldn't encourage others 3) there's a hell of an opportunity here Thoughts? Worth pursu…

As a person who loves tech writing, who owes his career to free coursera courses and online tutorials, and who is eager to teach people, this is an insanely difficult thing to get right.

Writing an ok tutorial isn't good enough. Writing an amazing tutorial is fine, if it is on a platform people know (such as LinuxAcademy, Pluralsight, or something similar).

I once wrote an article on getting started with a static website generator. I received a ton of praise in the comments, saying how great the step-by-step instructions are, and I felt great... Only to discover that I made a typo in one of the commands, and that if you actually went through the tutorial, there's no way you'd get past that one step, unless you knew what you were doing (in which case you wouldn't go through a getting started guide, most likely).

All I'm saying is, unless you can write an amazing content on a platform where people go to learn and advance their career, no one's gonna use it, I'm afraid.

Re: Kubernetes Failure Stories

#55
post #29

Earlier quoted context omitted.

Out of curiosity, why do you need RHEL or subscription from Canonical for the production Kubernetes setup? What's wrong with plain Ubuntu or CentOS?

It’s common to pay for things to make them easier to configure/manage. Red Hat OpenShift on RHEL, Pivotal Container Service on Ubuntu, Red Hat’s nextgen CoreOS based Kubernetes, Canonical’s Charmed Kubernetes Distribution on Ubuntu, etc. all have different config management , install, upgrade, patching mechanisms that vary from Ansible, to Terraform, to BOSH, to Juju. Some handle PXE bare metal, some don’t. Etc. Ther…

> It’s common to pay for things to make them easier to configure/manage.

Yes but in the case of RedHat specifically those goals are not achieved.

Re: Kubernetes Failure Stories

#56

Earlier quoted context omitted.

I'm consulting on a micro services back end right now with mostly prior experience with monoliths. What is the selling point that drives companies down this direction? It's insane, and my client keeps trying to hire new developers and bring on more consultants to build this thing, but the amount of knowledge required is more than any one person can handle. I have similar issues with their choice of db (nosql) and its…

The biggest reason for choosing microservices _should_ be scaling development teams: microservices allow multiple teams to work on different code bases, without stepping on each other's toes. What actually happens, tough, is that (uninformed) people choose it because they think it brings them scalability (wrong), it's more cloud compatible (wrong) or the worst offender, it's more modern.

If you achieve this scale of teams suggested, it means you have a lot of implicit gains here as well, right?

Such as higher quality, higher "velocity", separation of concern and hopefully a clear sense of ownership.

These are some of the things in my opinion that allows you to scale.

Re: Kubernetes Failure Stories

#57
post #12

Beyond strictly runtime failures, 2018 feels like the year that most of my friends tried kube but not everybody stayed on. The adoption failures are mostly networking issues specific to their cloud. Performance and box limits vary widely depending on cloud vendor and I still don't quite understand the performance penalty of the different overlay networks / adapters.

Network is a high performance system, and each layer you add adds latency. Consider a traditional monolithic application. In comes your HTTP request in one end, a bunch of cross thread communication happens, and database queries come out the other end. With that, you have 2 points of network communication. Now with a micro-service, you might have 4 or 5 applications that are needed to replace the above monolith. Thro…

Trusting your private network is private turns that whole network into a candy store once a beach head in that network has been established.

Defense in depth exists for a reason.

Re: Kubernetes Failure Stories

#58

Having used Docker Compose/Swarm for last two years, I remember having problems with them twice. One of which was an MTU setting which I didn't really understand why, but overall I was relatively happy with them. Since Kubernetes seems to have won, I decided to learn it but got some disappointments. The first disappointment is setting up a local development environment. I failed to get minikube running on a Macbook A…

I've had small-ish docker swarms in production for a couple of years as well, and I really don't understand why it doesn't seem to be popular at all. I feel like I need to move to K8S just because swarm seems to be going away, but I'm really not seeing the technical advantages at all.

If someone could point me to an article explaining why k8s is so much better than swarm, I'd really appreciate it. Are the big advantages only at 100-node scales?

Re: Kubernetes Failure Stories

#59
post #43

Having used Docker Compose/Swarm for last two years, I remember having problems with them twice. One of which was an MTU setting which I didn't really understand why, but overall I was relatively happy with them. Since Kubernetes seems to have won, I decided to learn it but got some disappointments. The first disappointment is setting up a local development environment. I failed to get minikube running on a Macbook A…

Digital Ocean's K8S offering is out of beta now: https://www.digitalocean.com/products/kubernetes/

DO K8s is pretty neat, but last time I checked it did not have the metrics server (CPU/mem metrics, also "kubectl top") yet.

Re: Kubernetes Failure Stories

#60

Having used Docker Compose/Swarm for last two years, I remember having problems with them twice. One of which was an MTU setting which I didn't really understand why, but overall I was relatively happy with them. Since Kubernetes seems to have won, I decided to learn it but got some disappointments. The first disappointment is setting up a local development environment. I failed to get minikube running on a Macbook A…

I've had small-ish docker swarms in production for a couple of years as well, and I really don't understand why it doesn't seem to be popular at all. I feel like I need to move to K8S just because swarm seems to be going away, but I'm really not seeing the technical advantages at all. If someone could point me to an article explaining why k8s is so much better than swarm, I'd really appreciate it. Are the big advanta…

I'm also constantly surprised at how unpopular docker swarm is given that everyone already uses docker itself. Why do you think swarm is going away though? I love the idea of just using my docker compose file as my deployment config.
Post reply on HN