Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Docker, Mesos, Marathon, and the End of Pets

blog.factual.com

Post reply on HN
Print view View original

31–36 of 36 posts

Re: Docker, Mesos, Marathon, and the End of Pets

#31

Post by jacques_chester »

steveb wrote: post #28
bkeroack wrote: post #23

Earlier quoted context omitted.

This is basically why Kubernetes exists: for all the plumbing, discovery, etc required on top of bare containers. It still requires work to go from zero to production-quality stack, of course.

We like Kubernetes (and are looking to add it to our project), but our goal is to integrate building blocks that allows us to run many different types of workloads. Think of our project more like an Ubuntu for distributed systems. Kubernetes may eventually spread out beyond Docker, but for today we need to support things like Kafka and Spark. As others have noted, we've had things like CloudFoundry, OpenShift and Her…

You should look at Cloud Foundry again, particularly with the introduction of Lattice. It used to be tied to apps, now it basically thinks about tasks and processes in a completely generic way.

Re: Docker, Mesos, Marathon, and the End of Pets

#32

Post by justizin »

KaiserPro wrote: post #2

I think the main problem with all of these systems is that are just so damn complex. Docker, its great if you have no state. But then if you have no state shit is easy. Mapping images to highspeed storage securely and reliably is genuinely hard. (unless you use NFSv4 and kerberos) Mesos is just over kill for everything. How many people actually need shared image programs bigger than a 64 core machine with 512gigs of…

> Mesos (sic) is just over kill for everything.

That's definitely an opinion. :)

I have seen Mesosphere deployed with great success.

Insofar as state, this is one reason I'm not crazy about CoreOS - I feel more comfortable containerizing the application tier than the data tier, though both are certainly possible.

I'm really not eager to replace a highly tuned MySQL or Postgres machine with a container environment experiencing several levels of abstraction and redirection. I get frustrated enough trying to align partitions with block boundaries through RAID controllers.

But if you have 20 front-end app servers and 5 machines that run cron jobs, container services can help you to utilize your capacity much better. I can't say how many times I've worked somewhere that we desperately needed capacity, but didn't have the budget to expand until we cleaned up a bunch of machines that were vastly underutilized.

Anyway, Mesosphere isn't perfect, I have only even used it moderately, but there's a lot of tooling out there which we can use.

Def agree on the wierd theology of fleet, but also generally that it just doesn't do enough for me. It's way too much fucking trouble to say, "Run an http proxy on each physical machine".

Re: Docker, Mesos, Marathon, and the End of Pets

#33

Post by samkone »

jacques_chester wrote: post #8

Factual have done what lots of people do, which is invent the first 20% of a PaaS. PaaSes are awesome. They also, once you go past the basics, require enormous engineering effort. And that's the problem: engineering effort spent on curating your own homegrown PaaS is engineering effort not available for creating user value. 5 years ago rolling your own was a source of competitive advantage. Today you can get an insta…

I know two companies running running openshift on Mesos. The reason modern Paas aren't enough, is the factat some scale you're not just running web services. But also rather complex data pipelines involving distributed data systems like kafka, spark, cassandra, etc .. and a simple paas has issues handling those workloads. That's where Mesos shines. As for the user value, I consider having an uptime system, providing reliable and intelligent service based on data processing, by efficiently using your resources is has an important business value.

Re: Docker, Mesos, Marathon, and the End of Pets

#34

Post by jacques_chester »

samkone wrote: post #33
jacques_chester wrote: post #8

Factual have done what lots of people do, which is invent the first 20% of a PaaS. PaaSes are awesome. They also, once you go past the basics, require enormous engineering effort. And that's the problem: engineering effort spent on curating your own homegrown PaaS is engineering effort not available for creating user value. 5 years ago rolling your own was a source of competitive advantage. Today you can get an insta…

I know two companies running running openshift on Mesos. The reason modern Paas aren't enough, is the factat some scale you're not just running web services. But also rather complex data pipelines involving distributed data systems like kafka, spark, cassandra, etc .. and a simple paas has issues handling those workloads. That's where Mesos shines. As for the user value, I consider having an uptime system, providing…

I find Mesos the most interesting because it's a kind of mirror image of Lattice[1], particularly Diego[2][2][3]. Both of them push intelligence out to the edges, but in different ways.

Diego pushes the scheduling problem out to the executors themselves through an auction mechanism. Mesos delegates it back to the requestors, as I understand it.

The way it was described to me by Onsi Fakhouri is that Diego is "demand-driven", and Mesos is "supply-driven". Diego grew from the lessons learnt on Cloud Foundry v2, so it favours fast placement of requests over perfect fit. But in the fact of the network fallacies, it probably turns out to be a good approach anyhow.

By way of warning, it has been a few months since I read the relevant papers and my memory is fuzzy.

Edit: and it looks as though the auction mechanism was moved away from. Hm.

[1] http://lattice.cf/

[2] https://github.com/cloudfoundry-incubator/diego-design-notes

[2] https://www.youtube.com/watch?v=1OkmVTFhfLY (an excellent overview)

[3] https://www.youtube.com/watch?v=SSxI9eonBVs (an excellent update)

Re: Docker, Mesos, Marathon, and the End of Pets

#35

Post by DannoHung »

justizin wrote: post #32
KaiserPro wrote: post #2

I think the main problem with all of these systems is that are just so damn complex. Docker, its great if you have no state. But then if you have no state shit is easy. Mapping images to highspeed storage securely and reliably is genuinely hard. (unless you use NFSv4 and kerberos) Mesos is just over kill for everything. How many people actually need shared image programs bigger than a 64 core machine with 512gigs of…

> Mesos (sic) is just over kill for everything. That's definitely an opinion. :) I have seen Mesosphere deployed with great success. Insofar as state, this is one reason I'm not crazy about CoreOS - I feel more comfortable containerizing the application tier than the data tier, though both are certainly possible. I'm really not eager to replace a highly tuned MySQL or Postgres machine with a container environment exp…

The thing I don't get about the storage issue is this: Does volume mounting into a Docker container impact performance meaningfully at all? No? Can't you just punt on the migratable storage question until then? You'd not be any worse off than you are and doing upgrades to the db engine would be pretty easy, right? I dunno, maybe there's something I'm missing, block storage, what are you gonna do?

Re: Docker, Mesos, Marathon, and the End of Pets

#36

Post by sytse »

edgan wrote: post #29
sytse wrote: post #11

Earlier quoted context omitted.

What didn't you like about the Omnibus rpms?

Omnibus packages are a horrible idea. They are security issues waiting to happen. A great example is the Chef server rpm. It is a 500mb mini distribution in one package. It has copies of perl, python, Ruby, and Erlang in it. If any of these has a security vulnerability, I have to wait on the maintainer to release a new version, and hope it included the security fixes. They also tend to include things like python head…

We are aware that we'll have to patch any security issues and have done so reliably. I agree it is not ideal and we'll always be slower than the distribution packages. On the other hand the installation is much faster to perform (2 minutes instead of 10 pages of copy pasting) and we're able to ship with very secure settings for the integration points (sockets, etc.). But we recognize that some people will prefer native packages and are sponsoring work to make native Debian packages.
Post reply on HN