Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apache Mesos to be moved to Attic

lists.apache.org

Post reply on HN
Print view View original

51–60 of 139 posts

Re: Apache Mesos to be moved to Attic

#51

Post by marcinzm »

ForHackernews wrote: post #40
marcinzm wrote: post #38

Earlier quoted context omitted.

Kubernetes is engineered to solve the five hundred different problems that different people have in a way that works for them. If you don't do that then everyone will complain about their feature or some edge case being missing and not use the system (see comments on mesos in this thread). That's not over engineering, that's the required amount of engineering for this sort of system.

Yeah, k8s people keep telling me this. But also, k8s "secrets" are not, in fact, secret, you can't actually accept traffic from the web without some extra magic cloud load balancer (cf https://v0-3-0--metallb.netlify.app/ maybe eventually) or properly run anything stateful like a database (maybe soon). Forget covering "everyone's" use cases: From where I'm sitting, k8s is an insanely complicated system that does a mi…

Your points mostly only matter if you're running on bare metal. If you're in the cloud then you've got load balancers and databases covered by your cloud provider. I need Kubernetes to handle problems that I already have great solutions for. I want it to handle the problems that my cloud provider provides poor or very specialized (ie: lock in) solutions for. Which for me it does very well and a lot more easily than doing so without Kubernetes.

edit: Kubernetes secrets are also either good enough (ie: on par with Heroku) or your cloud provider has an actual proper KMS.

Re: Apache Mesos to be moved to Attic

#52

Post by orf »

ForHackernews wrote: post #40
marcinzm wrote: post #38

Earlier quoted context omitted.

Kubernetes is engineered to solve the five hundred different problems that different people have in a way that works for them. If you don't do that then everyone will complain about their feature or some edge case being missing and not use the system (see comments on mesos in this thread). That's not over engineering, that's the required amount of engineering for this sort of system.

Yeah, k8s people keep telling me this. But also, k8s "secrets" are not, in fact, secret, you can't actually accept traffic from the web without some extra magic cloud load balancer (cf https://v0-3-0--metallb.netlify.app/ maybe eventually) or properly run anything stateful like a database (maybe soon). Forget covering "everyone's" use cases: From where I'm sitting, k8s is an insanely complicated system that does a mi…

> It's great that k8s (maybe) solves hard problems that nobody except Google has, but it doesn't solve the easy problems that most people have.

But it does though. Your dissent is that you don’t understand what secrets are and think load balancers are “magic”?

Come on. Also see this[1]

1. https://www.nginx.com/products/nginx-ingress-controller/

Re: Apache Mesos to be moved to Attic

#53

Post by macksd »

benjaminwootton wrote: post #23

I went to see Mesos early in their life in their San Francisco office after a joint customer put us in touch. Never in my life did I meet such an arrogant group of people. First, they left us waiting in reception for an hour. They eventually took the meeting over lunch, where we had to watch them inhaling their free food. Some guy in plastic-leather trousers spent most of the hour lecturing us about all of the multi…

I haven't seen much overt rudeness like what you're describing, but it does mystify me how often I'll meet with another company to discuss a potential integration story that benefits us both, and instead of any engineers who can discuss technical considerations, they send multiple product managers who all want to present their own slide deck about why they're so great.

I know you're great - that's why we requested this meeting. I think we're great too. There's a reason I think our integration would be a win-win and a win for users. Can we actually talk about it now? Oh let's schedule another meeting so the fourth product manager who didn't show up for this one can share their slide deck too.

Re: Apache Mesos to be moved to Attic

#54

Post by dharmab »

jeffbee wrote: post #35
dharmab wrote: post #31

Earlier quoted context omitted.

Most famously, Siri (used to?) run on a very large scale Mesos deployment (10000s of nodes, much higher than Kubernetes can scale to). Unfortunately the original article is lost, but here's a summary: https://daringfireball.net/linked/2015/04/29/siri-apache-mes...

OK but what was the utilization? I'm not really sure K8s is state-of-the-art either. There are published research papers about very-large-scale clusters with 80%+ resource utilization.

In our production experience, utilization had far more to do with the service owners (or autoscalers/auto-tuners) correctly choosing the cgroups and CPU scheduler allocations, as well as the kernel settings for cgroup slicing and CPU scheduler. We had Mesos clusters with 3% utilization and have Kubernetes clusters with 95%+ utilization. But we also have Kubernetes clusters with <10% utilization.

Re: Apache Mesos to be moved to Attic

#55

Post by jeffbee »

bdd wrote: post #43
jeffbee wrote: post #26

Earlier quoted context omitted.

Are there examples of high-utilization, large-scale Mesos deployments? Mesos didn't even gain over-commit until 2015, so it seems like it was generally behind the state of the art.

Twitter. From generic caches to ad serving; from stream processing to video encoding, all high utilization applications of either one or multiple schedulable resources.

As MesosCon Twitter said their cluster utilization was between 20 and 30%.

Re: Apache Mesos to be moved to Attic

#56

Post by dharmab »

random3 wrote: post #42

It's sad, but expected. This is not about Apache, but a failed open-governance for commercial open-source from Mesosphere. It's not the case with Apache Spark nor Apache Beam, HBase, etc. Mesos (and many other Berkeley AMPLab efforts) had briliant ideas behind it and an elegant implemention that allowed for much more than what Kubernetes was desgined to. Kubernetes was supposed to be scheduler for Mesos and Google in…

I'm an engineer at one of those companies you mentioned who worked directly on the Mesos+DC/OS platform.

Mesosphere's support was pretty much nonexistent as far as we were concerned. We had major issues open for years without any significant action. They were never resolved. We had to solve most problems ourselves.

We got so fed up that a few of us worked late nights (often up to 2AM) on a bottom-up skunkworks project to replace Mesos with Kubernetes. That project was exceptionally successful. (I'd like to point out that we are not in the Bay area and are adults with families and small children- we hated Mesos so much we were willing to stay up anyway.)

In short, Mesos has a lot of great theory but we hated it in practice. Kubernetes has some theoretical flaws but it is better experience for practitioners.

Re: Apache Mesos to be moved to Attic

#57

Post by star-trek-fleet »

random3 wrote: post #42

It's sad, but expected. This is not about Apache, but a failed open-governance for commercial open-source from Mesosphere. It's not the case with Apache Spark nor Apache Beam, HBase, etc. Mesos (and many other Berkeley AMPLab efforts) had briliant ideas behind it and an elegant implemention that allowed for much more than what Kubernetes was desgined to. Kubernetes was supposed to be scheduler for Mesos and Google in…

> Mesos (and many other Berkeley AMPLab efforts) had briliant ideas behind it

Brilliant on paper.

The idea was influenced by MR workloads on Borg. Because MR is so large a workload pattern on Borg, that it benefits from having its own scheduler, and Borg also benefit from unnecessary meddling.

The idea was only brilliant for a very small number of human users, both inside Google and in the broader industry. In terms of CPU & memory usage, mesos' approach should always take great share, because workloads that in the category of needing their own scheduler always is the big ones.

But as an open source product, its adopters are far off from that group. So you can see that all early adopters of Mesos are large corps. I seldom hear any startup applaud Mesos.

Precisely, I have repeated many times when Mesos was still relevant, that its model makes it costly to get started. It will never succeed unless it start to package a product that can provide the scheduler part.

We all know what happened after K8s. K8s works because Borg already worked for 12+ years.

> an elegant implemention that allowed for much more than what Kubernetes was desgined to.

By design K8s is more capable then Mesos. Because K8s includes scheduler. And numerous other capability. Heck, excluding performance and scalability, K8s is a level a bove Borg in terms of feature and capability. Mesos is far behind Borg even, not mentioning K8s.

Mesos does offer better scalability, on paper. I had no experience though.

> Kubernetes was supposed to be scheduler for Mesos and Google invested in it.

I never heard of such plan.

Re: Apache Mesos to be moved to Attic

#58

Post by dharmab »

rad_gruchalski wrote: post #50
random3 wrote: post #42

It's sad, but expected. This is not about Apache, but a failed open-governance for commercial open-source from Mesosphere. It's not the case with Apache Spark nor Apache Beam, HBase, etc. Mesos (and many other Berkeley AMPLab efforts) had briliant ideas behind it and an elegant implemention that allowed for much more than what Kubernetes was desgined to. Kubernetes was supposed to be scheduler for Mesos and Google in…

> Everything shifted towards DCOS and everyone was pushed towards commercial licenses. This was then partially diluted when Microsoft decided to deploy DC/OS and, from what I've heard, pushed Mesosphere to open source large chunks of DC/OS.

And then ACS was pretty much abandoned not long after that.

Re: Apache Mesos to be moved to Attic

#59

Post by ForHackernews »

orf wrote: post #52
ForHackernews wrote: post #40

Earlier quoted context omitted.

Yeah, k8s people keep telling me this. But also, k8s "secrets" are not, in fact, secret, you can't actually accept traffic from the web without some extra magic cloud load balancer (cf https://v0-3-0--metallb.netlify.app/ maybe eventually) or properly run anything stateful like a database (maybe soon). Forget covering "everyone's" use cases: From where I'm sitting, k8s is an insanely complicated system that does a mi…

> It's great that k8s (maybe) solves hard problems that nobody except Google has, but it doesn't solve the easy problems that most people have. But it does though. Your dissent is that you don’t understand what secrets are and think load balancers are “magic”? Come on. Also see this[1] 1. https://www.nginx.com/products/nginx-ingress-controller/

Yes, yes, there's a million zillion Kubernetes ingress things, none of which are really good enough that anyone uses them without a cloud-provider LB in front of it. Also they only deal with HTTP/S traffic. Got other types of traffic you want to run? Too bad, tunnel it over HTTPS.

If you want a picture of the future of computing, imagine everything-over-HTTPS-on-k8s-on-AWS stomping on a human face forever.

Re: Apache Mesos to be moved to Attic

#60

Post by ForHackernews »

marcinzm wrote: post #51
ForHackernews wrote: post #40

Earlier quoted context omitted.

Yeah, k8s people keep telling me this. But also, k8s "secrets" are not, in fact, secret, you can't actually accept traffic from the web without some extra magic cloud load balancer (cf https://v0-3-0--metallb.netlify.app/ maybe eventually) or properly run anything stateful like a database (maybe soon). Forget covering "everyone's" use cases: From where I'm sitting, k8s is an insanely complicated system that does a mi…

Your points mostly only matter if you're running on bare metal. If you're in the cloud then you've got load balancers and databases covered by your cloud provider. I need Kubernetes to handle problems that I already have great solutions for. I want it to handle the problems that my cloud provider provides poor or very specialized (ie: lock in) solutions for. Which for me it does very well and a lot more easily than d…

ITT: K8s is great because it frees us from the tyranny of Big Cloud providers.

Also ITT: Oh, but of course k8s is totally unusable unless you buy it from a Big Cloud provider.

Post reply on HN