Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apache Mesos to be moved to Attic

lists.apache.org

Post reply on HN
Print view View original

111–120 of 139 posts

Re: Apache Mesos to be moved to Attic

#111

Post by crmd »

_jal wrote: post #97
benjaminwootton wrote: post #23

I went to see Mesos early in their life in their San Francisco office after a joint customer put us in touch. Never in my life did I meet such an arrogant group of people. First, they left us waiting in reception for an hour. They eventually took the meeting over lunch, where we had to watch them inhaling their free food. Some guy in plastic-leather trousers spent most of the hour lecturing us about all of the multi…

> I had a similar experience with other west coast tech companies. I am the infra lead for a "west coast tech company", and they gave us the same guff. I knew we would not be using them after the first few minutes of listening to them. But please do continue to hate on this coast, tell your friends. I don't want to compete with more folks for housing.

I understand how Parent’s “west coast company” tag can come off as snotty, but I’ve begun using “west coast” and “east coast” as non-pejorative shorthand when discussing with a mixed audience the two major distinct and incompatible corporate operating systems in the tech industry.

The coastal definition is obviously imprecise, e.g. HP, based in Palo Alto, is as much an “east coast company” as IBM and JPMorgan Chase in New York, however it gets the point across more effectively and non-pejoratively IMO than enterprise vs startup, old school vs new school, companies who run on VMware vs k8s companies, oracle-y or Googley, boomer company vs hipster company, etc. I’ve found every person you’re talking to, whether they’re a developer, salesperson, HR recruiter, investor, etc. immediately knows what you’re taking about in context.

Long response but I had previously been thinking about this and appreciate to hear others’ thoughts.

Re: Apache Mesos to be moved to Attic

#112

Post by bogomipz »

dharmab wrote: post #9
xyzzy_plugh wrote: post #6

I used Mesos for a few years before experiencing Kubernetes. As neat as Mesos was, it was doomed from the start. For one, Kubernetes was, at least to some extent, a rewrite and extraction of functionality built at Google, from their production orchestration ecosystem that is Borg. The fact that Kubernetes was heavily influenced by a successful, large solution in this space allowed it to leapfrog, at least a bit, the…

We used Mesos in production until 2020 (started transition to Kubernetes in 2018), and this comment is incredibly accurate. Mesos was an interesting project but the defaults were incredibly naive about production environments. Two concrete examples: Mesos maintenance mode vs Kubernetes' cordoning and eviction APIs, and Mesos's default behavior when a Node is suddenly powered off vs Kubernetes'.

>"Two concrete examples: Mesos maintenance mode vs Kubernetes' cordoning and eviction APIs, and Mesos's default behavior when a Node is suddenly powered off vs Kubernetes'."

What was Mesos default behavior when a node was powered off?

Re: Apache Mesos to be moved to Attic

#113

Post by brutus1213 »

ghaff wrote: post #44
hintymad wrote: post #29

Earlier quoted context omitted.

A challenge with Mesos is that Mesos was a piece of technology, a framework at most, instead of a product. When I was using Mesos, the selling point was flexible and efficient resource scheduling. Unfortunately, resource/machine efficiency alone does not sell well, as most of the companies and individuals have betters things to worry about, say, productivity.

>the selling point was flexible and efficient resource scheduling There was a period when it wasn't clear that you didn't need both resource management and container orchestration. One of my colleagues was quite convinced at the time that we needed both Mesos and Kubernetes. If course, the market coalesced around Kubernetes which largely backfilled the missing capabilities.

Curious how you define resource manager. Is that the scheduler or something else?

Re: Apache Mesos to be moved to Attic

#114

Post by bogomipz »

benjamin_mahler wrote: post #71

I’m one of the long term PMC / committers on mesos. In retrospect I feel this was inevitable to a few key reasons: * k8s was a second system with all the learnings and experience of building such a system at Google for over a decade. Mesos was birthed by grad students and subsequently evolved into its position at Twitter but the engineers driving the project (myself included) did not have experience building cluster…

>"k8s was a second system with all the learnings and experience of building such a system at Google for over a decade. Mesos was birthed by grad students and subsequently evolved into its position at Twitter ..."

Do you know does Twitter still run Mesos?

Re: Apache Mesos to be moved to Attic

#115

Post by yujie1984 »

bogomipz wrote: post #106
yujie1984 wrote: post #102

Earlier quoted context omitted.

Well said Ben. I am also one of the long term PMC/Committer for the project. One of the lesson I learnt was that Mesos's two level resource allocations was originally designed for running batch workloads (e.g., spark, mpi, etc.) if you look at the original paper. Use it to run long running services is actually an after thought. We end up finding that we have to do lots of tuning on the first level scheduling algorith…

Thanks for the historical perspective. Might you or anyone else be able to recommend any resources that discuss the efforts to tune the two level scheduler for long-running workloads? You mentioned: >"The solution to the problem is actually optimistic offer, which is essentially the k8s model." Isn't the K8s model more "choose your QoS model" - BestError, Burstable or Guaranteed? Or am I misunderstanding your comment…

> Isn't the K8s model more "choose your QoS model" - BestError, Burstable or Guaranteed? Or am I misunderstanding your comment completely?

k8s's scheduling model is that scheduler is able to see the entire state of the cluster, thus can optimistically make optimal decisions on scheduling, especially for those long running jobs that are very picky in practice. Although k8s by default only runs the default scheduler, you could in theory run multiple schedulers in parallel (the omega model).

Mesos's pessimistic two level offer model makes it hard for second level scheduler to make optimal decisions because it might not get the right offer it needs. At the same time, first level scheduler lacks application specific information to make the right decision to send the right offer to the second level scheduler, thus the problem. We evaluated many first level scheduling algorithms, and ironically found that "random" first level scheduler sometimes works better than DRF for long running services scheduling.

> Could you say what was unique about the Mesos threading model?

Mesos uses a component called libprocess (think of it as C++ version of erlang). Each actor in the system (mesos master, mesos agent) is single threaded. Thus, all i/o operations need to be non-blocking to not block the actor. This makes it hard to integrate 3rdparty C++ libraries, especially those that involves I/O as they might have a different threading model.

Golang solved this problem using go-routing and bake that into the language. So the golang libraries, especially those involve I/O, are much more composable than C++ IMO.

Re: Apache Mesos to be moved to Attic

#116

Post by dragonsh »

0xEFF wrote: post #20
dragonsh wrote: post #7

Earlier quoted context omitted.

For over 90% of workloads kubernetes is an overkill. Only when company is reaching google scale kubernetes make sense. A good alternative to kubernetes is LXD [1] or just stick with docker compose. Kubernetes except for managed services from cloud providers is more difficult than an average application to manage and a huge cost in itself to run and maintain. [1] https://www.linuxcontainers.org/

> For over 90% of workloads kubernetes is an overkill. It's not. Take any simple web app and deploy it into managed GKE with Anthos and you automatically get SLI's and the ability to define SLO's for availability and latency with a nice wizard. Takes a few minutes to setup. The amount of engineering needed to achieve good SLO monitoring dwarfs the engineering needed to run a simple app so it just never happened. That…

Your statement already support that without the blessings of engineering team of Amazon, Google, Microsoft, Digital Ocean and various managed kubernetes service it's impossible for a reasonable small team to manage and monitor k8s and all of this service comes with lock-in and additional capital outlay.

Obviously for a Google Cloud Partner, more people are tied to gcp and kubernetes, higher the revenue. Its secondary if it's really necessary for an application to require k8s.

Re: Apache Mesos to be moved to Attic

#117

Post by andrecp11 »

ForHackernews wrote: post #60
marcinzm wrote: post #51

Earlier quoted context omitted.

Your points mostly only matter if you're running on bare metal. If you're in the cloud then you've got load balancers and databases covered by your cloud provider. I need Kubernetes to handle problems that I already have great solutions for. I want it to handle the problems that my cloud provider provides poor or very specialized (ie: lock in) solutions for. Which for me it does very well and a lot more easily than d…

ITT: K8s is great because it frees us from the tyranny of Big Cloud providers. Also ITT: Oh, but of course k8s is totally unusable unless you buy it from a Big Cloud provider.

The K8s docs offer ways of running on-prems, you can do it with nginx ingress daemonset + DNS pointing to the IP (or CNAME) of your workers.

It's all covered quite well here: https://kubernetes.github.io/ingress-nginx/deploy/baremetal/ and we use it in production.

Re: Apache Mesos to be moved to Attic

#119

Post by audi0slave »

bogomipz wrote: post #114
benjamin_mahler wrote: post #71

I’m one of the long term PMC / committers on mesos. In retrospect I feel this was inevitable to a few key reasons: * k8s was a second system with all the learnings and experience of building such a system at Google for over a decade. Mesos was birthed by grad students and subsequently evolved into its position at Twitter but the engineers driving the project (myself included) did not have experience building cluster…

>"k8s was a second system with all the learnings and experience of building such a system at Google for over a decade. Mesos was birthed by grad students and subsequently evolved into its position at Twitter ..." Do you know does Twitter still run Mesos?

AFAIK, it does. Uber does too, though both of them are moving to k8s

source: ex uber compute team member

Re: Apache Mesos to be moved to Attic

#120

Post by IgorPartola »

benjamin_mahler wrote: post #71

I’m one of the long term PMC / committers on mesos. In retrospect I feel this was inevitable to a few key reasons: * k8s was a second system with all the learnings and experience of building such a system at Google for over a decade. Mesos was birthed by grad students and subsequently evolved into its position at Twitter but the engineers driving the project (myself included) did not have experience building cluster…

I think the way you explained your last point really hits the nail on the head in terms of FOSS. I did actually enjoy large parts of Revolution OS, the movie about the creation of GNU and Linux, but the part that stood out to me the most was cmdrtaco explaining open source (paraphrasing here): at the end of the day you end up working on something you need, and then you think “if I need this maybe someone else does to” so you publish the source code and let others use it. This stuck with me because, well, if I publish something I found useful and nobody else finds it useful, oh well. But if they do, that’s really great! I am not saying Google open sourced k8s out of sheer goodness of their hearts, but I think it’s a lot harder to maintain that sensibility when the project is VC backed.
Post reply on HN