Apache Mesos to be moved to Attic • Hacker News Board

Re: Apache Mesos to be moved to Attic

#31

Post by dharmab » Tue, Apr 06, 2021, 5:11 PM UTC

jeffbee wrote: post #26

kraemate wrote: post #16

Mesos made a ton of new contributions to distributed resource management. Resource offers and application integration can allow high cluster utilization and efficiency. Giving applications the opportunity to take part in resource allocation and scheduling was similar to the Exokernel design, and led to many interesting middleware architectures. Mesos also introduced Dominant Resource Fairness for allocating resources…

Are there examples of high-utilization, large-scale Mesos deployments? Mesos didn't even gain over-commit until 2015, so it seems like it was generally behind the state of the art.

Most famously, Siri (used to?) run on a very large scale Mesos deployment (10000s of nodes, much higher than Kubernetes can scale to).

Unfortunately the original article is lost, but here's a summary: https://daringfireball.net/linked/2015/04/29/siri-apache-mes...

Re: Apache Mesos to be moved to Attic

#32

Post by zekrioca » Tue, Apr 06, 2021, 5:11 PM UTC

fnord123 wrote: post #19

Fun fact: According to the initial Mesos paper: Spark was a demo project to show off Mesos. https://people.eecs.berkeley.edu/~alig/papers/mesos.pdf > To validate our hypothesis that specialized frameworks providevalue over general ones, we have also built a new frame-work on top of Mesos called Spark ...

Look at the authors list. Some of them also participated in creating and designing Spark.

Re: Apache Mesos to be moved to Attic

#33

Post by fnord123 » Tue, Apr 06, 2021, 5:14 PM UTC

zekrioca wrote: post #32

fnord123 wrote: post #19

Fun fact: According to the initial Mesos paper: Spark was a demo project to show off Mesos. https://people.eecs.berkeley.edu/~alig/papers/mesos.pdf > To validate our hypothesis that specialized frameworks providevalue over general ones, we have also built a new frame-work on top of Mesos called Spark ...

Look at the authors list. Some of them also participated in creating and designing Spark.

Yes, this is not a surprise. The fun fact part is that Spark began as merely a way to validate the hypothesis of Mesos and look how the hypothesis validation project has taken off. Neat!

Re: Apache Mesos to be moved to Attic

#34

Post by ForHackernews » Tue, Apr 06, 2021, 5:14 PM UTC

mplewis wrote: post #18

dragonsh wrote: post #7

Earlier quoted context omitted.

For over 90% of workloads kubernetes is an overkill. Only when company is reaching google scale kubernetes make sense. A good alternative to kubernetes is LXD [1] or just stick with docker compose. Kubernetes except for managed services from cloud providers is more difficult than an average application to manage and a huge cost in itself to run and maintain. [1] https://www.linuxcontainers.org/

For a different perspective, learning Kubernetes and using it widely gives you a universal set of tools for managing all sorts of applications at all scales. https://news.ycombinator.com/item?id=26502900

Sure, this is kind of true, but only in the most depressing way possible. Kubernetes is overengineered and terrible but it's also just about the only game in town if you want a vendor-agnostic container orchestration system.

This is the same situation as Javascript circa 2008: "Learning this absolute dogshit language and using it widely will give you the ability to write universal applications that will run in browsers everywhere and make you very employable."

You're not wrong about k8s today, and wouldn't have been wrong about JS in the past, but boy is it a sad indictment of our industry that these things are true.

Re: Apache Mesos to be moved to Attic

#35

Post by jeffbee » Tue, Apr 06, 2021, 5:15 PM UTC

dharmab wrote: post #31

jeffbee wrote: post #26

Earlier quoted context omitted.

Are there examples of high-utilization, large-scale Mesos deployments? Mesos didn't even gain over-commit until 2015, so it seems like it was generally behind the state of the art.

Most famously, Siri (used to?) run on a very large scale Mesos deployment (10000s of nodes, much higher than Kubernetes can scale to). Unfortunately the original article is lost, but here's a summary: https://daringfireball.net/linked/2015/04/29/siri-apache-mes...

OK but what was the utilization? I'm not really sure K8s is state-of-the-art either. There are published research papers about very-large-scale clusters with 80%+ resource utilization.

Re: Apache Mesos to be moved to Attic

#36

Post by marcinzm » Tue, Apr 06, 2021, 5:17 PM UTC

rantwasp wrote: post #13

marcinzm wrote: post #11

Earlier quoted context omitted.

>except for managed services from cloud providers I'd imagine most small to medium companies would be running things on a cloud service using managed kubernetes. It seems mostly larger companies that are sticking with non-cloud hardware and services. The advantage of kubernetes in that case is that there is a large ecosystem of helm charts, guides, documentation, etc. Deploying something new from scratch is fairly ea…

just because something is packaged does not mean it’s usable. YMMV but this is how security horror stories start. Someone ran a container they had no idea where it came from, happily used a helm chart. Most of the times it’s not even malicious - it’s outdated software because “it just works”

In my experience all the common helm charts and docker images are regularly updated. If you don't update your installation of them then you also wouldn't update a docker compose or LXD.

Re: Apache Mesos to be moved to Attic

#37

Post by kbar13 » Tue, Apr 06, 2021, 5:19 PM UTC

desktopninja wrote: post #14

fiberoptick wrote: post #2

End of an era... Are there any viable alternatives to Kubernetes and Nomad?

Personally think AWS ECS is a 3rd place contender. Would add sprinkles to it if only they'd allow yaml files vs json configs in the aws-cli. ecs-cli and copilot are alright. Generally prefer to stay as close as possible to aws-cli

AWS CDK makes ECS a lot more palatable

Re: Apache Mesos to be moved to Attic

#38

Post by marcinzm » Tue, Apr 06, 2021, 5:20 PM UTC

ForHackernews wrote: post #34

mplewis wrote: post #18

Earlier quoted context omitted.

For a different perspective, learning Kubernetes and using it widely gives you a universal set of tools for managing all sorts of applications at all scales. https://news.ycombinator.com/item?id=26502900

Sure, this is kind of true, but only in the most depressing way possible. Kubernetes is overengineered and terrible but it's also just about the only game in town if you want a vendor-agnostic container orchestration system. This is the same situation as Javascript circa 2008: "Learning this absolute dogshit language and using it widely will give you the ability to write universal applications that will run in browse…

Kubernetes is engineered to solve the five hundred different problems that different people have in a way that works for them. If you don't do that then everyone will complain about their feature or some edge case being missing and not use the system (see comments on mesos in this thread). That's not over engineering, that's the required amount of engineering for this sort of system.

Re: Apache Mesos to be moved to Attic

#39

Post by jaaron » Tue, Apr 06, 2021, 5:29 PM UTC

I preferred Mesos to k8s. I think it's core architecture (a 2-level scheduler) is a better foundation. For the longest time, I felt k8s was effectively an overgrown hobby project that had no place being deployed the way it was. That had me realize something in the shower this morning:

k8s is the Rails of the cloud.

Back when Rails came out, it too was a bit of a hobby project. When coming from more established enterprise web frameworks, Rails felt like a toy. It didn't have the features, robustness, safety, and scalability of "proper" frameworks.

What did Rails do? It was easy to get started and it hid a lot of the boring and painful work of web frameworks at the time.

Through the sheer force of will of a massive community, Rails grew up and became something more the toy it started as. I was pretty arrogant in my opinions about the Rails community at first, but then I ended up working on several Rails projects over the years.

It still hides a lot under the hood, there are still arguably better technical frameworks out there and plenty of folks use it improperly, when they don't need to, and without really understanding the fundamentals, meaning that they tend to get in trouble when pushing the limits or moving outside the golden path of development.

And I feel the same way about k8s. I think it started out without anywhere near the features of similar frameworks. It didn't scale well, was simplistic in it's model, and overly complicated in it's implementation. But it was much more approachable than something like Mesos and answered the question of "why am I containerizing everything?", giving everything a purpose to those who started down the path of Docker. And now it has a huge following and industry behind it.

At this point, I've learned that what becomes popular isn't necessarily the "right" or "correct" architecture. There's a lot more to programming trends (and fads) than that. The whole industry seems to want to reinvent the wheel every decade, almost like some sort of planned obsolescence to justify our work. Nevertheless, it's rarely wise to fight against the tide and when you have the enough of the industry moving in a direction, we can make even toys into real tools.

Re: Apache Mesos to be moved to Attic

#40

Post by ForHackernews » Tue, Apr 06, 2021, 5:31 PM UTC

marcinzm wrote: post #38

ForHackernews wrote: post #34

Earlier quoted context omitted.

Sure, this is kind of true, but only in the most depressing way possible. Kubernetes is overengineered and terrible but it's also just about the only game in town if you want a vendor-agnostic container orchestration system. This is the same situation as Javascript circa 2008: "Learning this absolute dogshit language and using it widely will give you the ability to write universal applications that will run in browse…

Kubernetes is engineered to solve the five hundred different problems that different people have in a way that works for them. If you don't do that then everyone will complain about their feature or some edge case being missing and not use the system (see comments on mesos in this thread). That's not over engineering, that's the required amount of engineering for this sort of system.

Yeah, k8s people keep telling me this.

But also, k8s "secrets" are not, in fact, secret, you can't actually accept traffic from the web without some extra magic cloud load balancer (cf https://v0-3-0--metallb.netlify.app/ maybe eventually) or properly run anything stateful like a database (maybe soon).

Forget covering "everyone's" use cases: From where I'm sitting, k8s is an insanely complicated system that does a miserable job of covering the 95% use cases that Heroku solved in 2008.

It's great that k8s (maybe) solves hard problems that nobody except Google has, but it doesn't solve the easy problems that most people have.