Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kubernetes Failure Stories

srcco.de

Post reply on HN
Print view View original

121–130 of 242 posts

Re: Kubernetes Failure Stories

#121

Post by jordanbeiber »

mooreds wrote: post #118
jordanbeiber wrote: post #37

Earlier quoted context omitted.

Microservices requires proper, structured systems management. Most seem to think "devops" and microservices are ways to ignore this, when in fact the complete opposite applies. Make sure you have a true service delivery and service management pipeline in place where it is real easy for dev and ops to deploy and decommission services. Service metadata is key in my experience. Not too much though, just enough (such as…

> Granted, this was within a multibillion turnaround enterprise. > Most businesses would benefit from this transformation... That is a great story from the front lines. Thank you for sharing. My worry is when I see small companies or startups using microservices. That seems nutty to me

The money bit was more about rolling a change that means instant revenue increase in the millions, rather than people or product scale, but I agree that you need to put some serious effort into service management that might not make sense for a really small team.

Not sure where there’s a natural threshold other than when you realize you can’t deliver on business needs and requirements rapidly enough.

This is probably not in the beginning with a small team.

Re: Kubernetes Failure Stories

#122

Post by devthane »

pquerna wrote: post #111
dankohn1 wrote: post #40

Earlier quoted context omitted.

> The only easy and reliable ways to have a production grade Kubernetes cluster are to lock yourself into either a big player cloud provider, or an enterprise OS (Redhat/Ubuntu), or introduce a new layer on top of Kubernetes [1] I think you might have misunderstood that page. The standard and universal way to deploy Kubernetes on to either your own bare metal or any cloud provider is to use kubeadm. However, if you w…

The point is not about the minimum conformance, but rather the lock-in provided by the maximum configuration / extensions of each vendor. Take AWS EKS as an example. Their feature page[1] does mention conformance. Then it mentions 20 other non-conformance focused features that create an effective lock-in. k8s is becoming like OpenStack in this regards. You need to embrace a vendor version of k8s in order to have a fu…

This isn’t my experience at all. I as one person taught myself over the past couple years docker then Kubernetes and am now managing a small 3 node bare metal cluster on my own.

But using rancher 2.0 has helped a bunch to ease me into it. Now I feel comfortable enough to start up a cluster on my own without it.

Re: Kubernetes Failure Stories

#123

Post by lmm »

jordanbeiber wrote: post #117
lmm wrote: post #109

Earlier quoted context omitted.

Automated deployment and the like is worthwhile whether your system is microservice or monolith. But no amount of automation can eliminate the overhead a network boundary brings to local development.

We view this a bit different, and that’s fine (not the automation part, here we agree). Automation in this context is for me more than just the deploy bit, it’s also about testing and service management which includes for example service relationships and discovery. If you could do local dev on that app as a monolith, it can most likely be done broken up in smaller services as well. There are no silver bullets to be…

> If you could do local dev on that app as a monolith, it can most likely be done broken up in smaller services as well.

It's possible, but the overhead is a lot higher, and that weighs down everything you do. Your edit-test cycle gets longer, development gets slower.

> There are no silver bullets to be had anywhere, right? Just use whatever processes and tools that work, until they don’t I guess.

Nothing is perfect but often one choice is better than another. I've seen microservices go badly much more often than monoliths, and most successful microservice systems were built as a monolith first with services separated out only when it became necessary.

Re: Kubernetes Failure Stories

#124

Post by manigandham »

pgwhalen wrote: post #116
manigandham wrote: post #110

I don't understand all the negative comments here, K8S solves many problems regardless of scale. You get a single platform that can run namespaced applications using simple declarative files with consolidated logging, monitoring, load-balancing, and failover built-in. What company would not want this?

I very much agree that kubernetes is useful in an environment that doesn’t need to scale, but do tell how it enables consolidated logging and monitoring, since my medium/small shop is spending quite some time setting up our own infrastructure for it.

Are you running it yourself? The K8S dashboard gives you logs and basic monitoring out of the box, or you can get logs directly from kubectl.

Re: Kubernetes Failure Stories

#125

Post by manigandham »

mirceal wrote: post #115
manigandham wrote: post #110

I don't understand all the negative comments here, K8S solves many problems regardless of scale. You get a single platform that can run namespaced applications using simple declarative files with consolidated logging, monitoring, load-balancing, and failover built-in. What company would not want this?

this is too broad. i think that may actually be the problem: in theory it can do a lot of things, but in the real world it’s hard to get all those theoretical benefits. for me, if you’re in the cloud you don’t need k8s. your favorite cloud provider has already figured out logging and monitoring and the basic things you need to get going. (another story if you run on bare metal) if you’re not running a legacy app you…

There is a very big gap between IaaS and PaaS. K8S is an abstraction on top of VMs so you can have a customizable PaaS that runs on YAML code. It has nothing to do with how complex your app is because K8S is about running it with less work in a declarative fashion. I'm currently in and have worked with dozens of startups that have saved lots of time by removing all the ops overhead with K8S because it runs the servers and we can just deploy our apps.

It seems like most of the problems are actually about installing and running K8S software itself, but then 95% of companies won't be doing that and using the managed offerings instead. This is no different than companies using the cloud over running their own DCs.

Re: Kubernetes Failure Stories

#126

Post by verisimilidude »

m0zg wrote: post #51

It's not for everyone and it has significant maintenance overhead if you want to keep it up to date _and_ can't re-create the cluster with a new version every time. This is something most people at Google are completely insulated from in the case of Borg, because SRE's make infrastructure "just work". I wish there was something drastically simpler. I don't need three dozen persistent volume providers, or the ability…

Simplified Kubernetes is a thing that exists. OpenShift (and the open source version, OKD) jumps out as the immediate example. There are other non-k8s tools that cover some of the same territory, like Docker Swarm or Cloud Foundry.

There's still a learning curve, but it's much more humane than Kubernetes.

Re: Kubernetes Failure Stories

#127

Post by sonnyblarney »

m0zg wrote: post #108
sonnyblarney wrote: post #105

Earlier quoted context omitted.

No, there is NIMBY-ism in most places, the reason Valley prices are sky high is the salaries. " There's no reason a dilapidated teardown-ready shack should cost $2M, no matter where it is." The cost is not the shack, it's the land it's sitting on. The higher the salaries in the valley, the more that dilapidated shack will cost.

You make it sound like people enjoy paying millions for dilapidated shacks, which I can assure you is not the case. The reason housing costs so much is limited supply coupled with high demand. There could be a 10-story 40-apartment building in place of that one shack. Place a few thousand of those strategically through Bay Area, and the price per square foot would come down big time even if the cost of land stays hig…

" much is limited supply coupled with high demand. "

No, it's just 'supply and demand' neither are 'high or low' necessarily.

SV has quite high wages, that's a huge driver of demand.

The residents of SV do not want to be like NY or Hong Kong, that is their choice. It's the choice many, many places make as well. Zurich, Paris, even London, they don't live in high rises.

The attractiveness of Cali in many ways is that it's not entirely flooded/urban like NYC.

There are not just arbitrary ways to create more homes, it has an effect on the situation.

Re: Kubernetes Failure Stories

#128

Post by jacques_chester »

cygned wrote: post #86
jacques_chester wrote: post #79

Earlier quoted context omitted.

Kubernetes has always had an identity crisis. Who is aimed at, app developers or platform operators? Clear, obvious contracts between the two roles are valuable, even if you decide to combine them. I'm moderately hopeful that Knative will help in that regard, as it is more conclusively oriented towards the developer. But I am wary that since it leaves the implementation details completely visible, it may not achieve…

Achieving that is a difficulty task, though. Personally, I’d not like to rely on an abstraction on top of a system with that level of complexity for production because I expect to run into situations that can only be solved with deep knowledge of k8s.

I feel like this separation is achievable, if only because it's been achieved several times already.

Google App Engine did it. Heroku did it. Cloud Foundry did it. In none of these situations does an app developer need to know or care how the bits are plumbed, they only need to enjoy hot and cold running code.

Re: Kubernetes Failure Stories

#129

Post by jacques_chester »

jordanbeiber wrote: post #117
lmm wrote: post #109

Earlier quoted context omitted.

Automated deployment and the like is worthwhile whether your system is microservice or monolith. But no amount of automation can eliminate the overhead a network boundary brings to local development.

We view this a bit different, and that’s fine (not the automation part, here we agree). Automation in this context is for me more than just the deploy bit, it’s also about testing and service management which includes for example service relationships and discovery. If you could do local dev on that app as a monolith, it can most likely be done broken up in smaller services as well. There are no silver bullets to be…

In think your interlocutor's point is that type checking by a compiler is a lot simpler, more reliable and more performant than any networked service discovery scheme so far conceived.

I think microservice architectures can genuinely decouple teams to iterate faster and consolidate efforts. But it's not a free lunch.

Re: Kubernetes Failure Stories

#130

Post by pgwhalen »

manigandham wrote: post #124
pgwhalen wrote: post #116

Earlier quoted context omitted.

I very much agree that kubernetes is useful in an environment that doesn’t need to scale, but do tell how it enables consolidated logging and monitoring, since my medium/small shop is spending quite some time setting up our own infrastructure for it.

Are you running it yourself? The K8S dashboard gives you logs and basic monitoring out of the box, or you can get logs directly from kubectl.

We are. I guess I wouldn’t consider that enough for our purposes. Actually retaining and alerting on logs, or alerting on anything for that matter is not out of the box unless I’m missing something.
Post reply on HN