Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kubernetes Failure Stories

srcco.de

Post reply on HN
Print view View original

141–150 of 242 posts

Re: Kubernetes Failure Stories

#141

Post by erulabs »

pgwhalen wrote: post #116
manigandham wrote: post #110

I don't understand all the negative comments here, K8S solves many problems regardless of scale. You get a single platform that can run namespaced applications using simple declarative files with consolidated logging, monitoring, load-balancing, and failover built-in. What company would not want this?

I very much agree that kubernetes is useful in an environment that doesn’t need to scale, but do tell how it enables consolidated logging and monitoring, since my medium/small shop is spending quite some time setting up our own infrastructure for it.

I run a Filebeat container with privileges to read stdout/stderr of all other pods, which then forwards to ElasticSearch. (https://www.elastic.co/guide/en/beats/filebeat/master/runnin...). It's fairly straight forward, then Kibana + Watcher can ship alters to PagerDuty based on log patterns / queries / limits, etc. I think Watcher is open-source/free now?

I also have Prometheus + grafana, which similarly collects lots of stats from around the cluster, but I'm fairly sure I'm the only person who uses that dashboard, since the only things hooked up to Prometheus are databases and such, no internal applications (yet!).

Being able to aggregate stdout/stderr across dozens of machines previously would have cost either a lot of Chef setup time or a contract with some provider. Now I get a fairly straight forward open-source stack that can be refined over time, and the yaml re-used very easily in any cluster. Plus, the metadata collected from Kubernetes about each log line is extremely useful (For example, out of the box you can query by Kubernetes labels for your graphs etc)

Re: Kubernetes Failure Stories

#142

Post by zerotolerance »

DavidWoof wrote: post #58
dvnguyen wrote: post #21

Having used Docker Compose/Swarm for last two years, I remember having problems with them twice. One of which was an MTU setting which I didn't really understand why, but overall I was relatively happy with them. Since Kubernetes seems to have won, I decided to learn it but got some disappointments. The first disappointment is setting up a local development environment. I failed to get minikube running on a Macbook A…

I've had small-ish docker swarms in production for a couple of years as well, and I really don't understand why it doesn't seem to be popular at all. I feel like I need to move to K8S just because swarm seems to be going away, but I'm really not seeing the technical advantages at all. If someone could point me to an article explaining why k8s is so much better than swarm, I'd really appreciate it. Are the big advanta…

Swarm is not going anywhere.

Re: Kubernetes Failure Stories

#143

Post by shereadsthenews »

wikibob wrote: post #73
dilyevsky wrote: post #69

Earlier quoted context omitted.

Re configuration: ksonnet is an option (although I personally find jsonnet a “lipstick on a pig” kind of solution). There’s some work going on to have something more user-friendly (think Google’s Piccolo) - https://github.com/stripe/skycfg (disclaimer - I contributed to this project)

Could you describe Piccolo a bit? Can't find anything on it.

Piccolo is very similar to Pystachio.

Re: Kubernetes Failure Stories

#144

Post by shaklee3 »

m0zg wrote: post #51

It's not for everyone and it has significant maintenance overhead if you want to keep it up to date _and_ can't re-create the cluster with a new version every time. This is something most people at Google are completely insulated from in the case of Borg, because SRE's make infrastructure "just work". I wish there was something drastically simpler. I don't need three dozen persistent volume providers, or the ability…

Quite the opposite, I want it to be flexible and pluggable for other use cases other than the most simple. I've gotten a lot of benefit from adding custom features.

Re: Kubernetes Failure Stories

#145

Post by 013a »

stunt wrote: post #50

Kubernetes solves a problem that most of the companies don't have. That is why I don't understand why the hype around it is so big. For the majority, it just adds a little value when you compare to added complexity to infrastructure and the cost of a learning curve and the ongoing operation and maintenance.

I disagree, almost entirely. Kubernetes solves problems that every single cloud-based software company has.

What's the alternative? We spin up VMs, templated with AMIs, provisioned with an ASG? That works fine. But we want centralized logging. We want graceful restarts. We want automated rollbacks. The list goes on. These are not Google scale desires, these are "cost of doing business" asks for any cloud company. You can start building all of this on that core architecture of AMIs, or your cloud provider's equivalent, but all you're going to do re-invent what Kubernetes does, probably worse.

Kubernetes' problem isn't that it solves problems most companies don't have. The problem is that these problems most companies have could be solved in a simpler way than Kubernetes, because most companies have the exact same problems.

Re: Kubernetes Failure Stories

#146

Post by shaklee3 »

cygned wrote: post #49

I am a developer and I find k8s frustrating. To me, its documentation is confusing and scattered among too many places (best example: overlay networks). I have read multiple books and gazillions of articles and yet I have the feeling that I am lacking the bigger picture. I was able to set it up successfully a couple of times, with more or less time required. Last time, I gave up after four days because I realized tha…

Kubernetes doesn't specify anything about overlay networks. That's up to the CNI provider. Are you referring to flannel's documentation?

Re: Kubernetes Failure Stories

#147

Post by aprdm »

m0zg wrote: post #51

It's not for everyone and it has significant maintenance overhead if you want to keep it up to date _and_ can't re-create the cluster with a new version every time. This is something most people at Google are completely insulated from in the case of Borg, because SRE's make infrastructure "just work". I wish there was something drastically simpler. I don't need three dozen persistent volume providers, or the ability…

In places worked we usually had a vmware cluster, load balancer, NFS for shared data when necessary and DNS set up (e.g: through consul).

This setup is very, very simple and scalable. There is very little to gain IMO on moving to Kubernetes.

Consul, VSphere and load balancers have APIs and you can write tools to do everything that K8s does.

Re: Kubernetes Failure Stories

#148

Post by plasma »

bdcravens wrote: post #34

I've started the planning phase of a Kubernetes course, geared toward developers more so than the enterprise gatekeepers. As I read stories like these, I jump between different thoughts and feelings: 1) no matter what I think I know, there's too many dark corners to create an adequate course 2) K8S is such a dumpster fire that I shouldn't encourage others 3) there's a hell of an opportunity here Thoughts? Worth pursu…

I think a good course would be setting up a k8s cluster for a simple "hello world" production app, which then includes topics perhaps about monitoring, upgrading, etc all the kind of stuff you want to know for getting an app up and running.

Re: Kubernetes Failure Stories

#149

Post by spronkey »

stingraycharles wrote: post #28
lucidone wrote: post #18

Earlier quoted context omitted.

I'm consulting on a micro services back end right now with mostly prior experience with monoliths. What is the selling point that drives companies down this direction? It's insane, and my client keeps trying to hire new developers and bring on more consultants to build this thing, but the amount of knowledge required is more than any one person can handle. I have similar issues with their choice of db (nosql) and its…

The biggest reason for choosing microservices _should_ be scaling development teams: microservices allow multiple teams to work on different code bases, without stepping on each other's toes. What actually happens, tough, is that (uninformed) people choose it because they think it brings them scalability (wrong), it's more cloud compatible (wrong) or the worst offender, it's more modern.

I personally feel like microservices don't necessarily assist in allowing multiple disconnected teams to work on different codebases. In my mind the reason is similar to why it's so difficult to reuse code - requirements are slightly different between two things, so you either design a service to handle both, turning it into mini rather than micro, or you have two services - in which case you're just adding more plumbing to what could have just been a separate monolith.

The "services" part makes a lot of sense in certain situations, and I feel like one of the best application architectures is what I call disconnected monoliths. Centralise and standardise core concerns - like authentication, external communications. Build monoliths for everything else.

Re: Kubernetes Failure Stories

#150

Post by manigandham »

romeisendcoming wrote: post #65
dvnguyen wrote: post #21

Having used Docker Compose/Swarm for last two years, I remember having problems with them twice. One of which was an MTU setting which I didn't really understand why, but overall I was relatively happy with them. Since Kubernetes seems to have won, I decided to learn it but got some disappointments. The first disappointment is setting up a local development environment. I failed to get minikube running on a Macbook A…

Or you could just learn how to manage infrastructure the old fashioned way, which was never broken for small business and mid-sized enterprise environments. The only time you need the complexity and overhead of something like kubernetes is when you are truly large or when you have caught the in-fashion disease. It's quite simple for a 20 year SA to stand up a highly integrated environment with modular monitoring, dir…

Why not hire an SA? Because K8S is free and runs well from small to large and is available on every cloud where the IT infrastructure already is.

Why is it better to spend money to rebuild a fraction of K8S with a patchwork of infrastructure put together by a single person?

Post reply on HN