Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Docker, Mesos, Marathon, and the End of Pets

blog.factual.com

Post reply on HN
Print view View original

21–30 of 36 posts

Re: Docker, Mesos, Marathon, and the End of Pets

#21

Post by nuschk »

steve0ps wrote: post #18
Wilya wrote: post #9

Is anyone running Marathon in production? Real production. The kind where any downtime means lost money. I see a lot of intro-level tutorials, but almost nothing on the more advanced side. My (completely casual) experience with Marathon is pretty bad, with the main process crashing quite regularly even under no load, so I'm wondering if people who write about these systems have actually used them for non-trivial task…

I've been running Marathon in production (real production) to power more than 100 applications for the past six months. I chose it because it seemed like the most stable thing at the time; however, quickly found it was not production ready. While many of the original issues I encountered in 0.7.x were resolved with the 0.8.x release, 0.8.x brought new issues such as stuck deployments, etc. Additionally, I have found…

Might I ask: If you're moving away from Marathon, where are you moving to? Kubernetes?

Re: Docker, Mesos, Marathon, and the End of Pets

#22

Post by nemothekid »

steve0ps wrote: post #18
Wilya wrote: post #9

Is anyone running Marathon in production? Real production. The kind where any downtime means lost money. I see a lot of intro-level tutorials, but almost nothing on the more advanced side. My (completely casual) experience with Marathon is pretty bad, with the main process crashing quite regularly even under no load, so I'm wondering if people who write about these systems have actually used them for non-trivial task…

I've been running Marathon in production (real production) to power more than 100 applications for the past six months. I chose it because it seemed like the most stable thing at the time; however, quickly found it was not production ready. While many of the original issues I encountered in 0.7.x were resolved with the 0.8.x release, 0.8.x brought new issues such as stuck deployments, etc. Additionally, I have found…

I wouldn't expect Marathon to do service discovery for you, as I believe that is better left to something like Mesos-dns/Consul which marathon can supervise for you, and docker integration has been fairly simple.

In any case I found my marathon was not without issues, like failover causing every application to restart (I think this was fixed in 0.8.2), or the fact that marathon tends to use 2x as much RAM as Zookeeper or Mesos-Master (I run the 3 on the same node).

Have you seen the aurora apache project? It solves the same problems as marathon, and its creators claim it was built to handle stability. I originally chose marathon as JSON configuration over REST was easier to wrap my head around, but was this something you tried and how did it work for you?

Re: Docker, Mesos, Marathon, and the End of Pets

#23

Post by bkeroack »

steveb wrote: post #12

The real problem is going from tutorial to something you would use in production. Throw in logging, security and service discovery and you can have a few engineers hacking away for months. So I want to plug a project I've been contributing to: https://github.com/CiscoCloud/microservices-infrastructure We're trying to make it super easy to deploy these tools. For example every time you launch a docker container, it wi…

This is basically why Kubernetes exists: for all the plumbing, discovery, etc required on top of bare containers.

It still requires work to go from zero to production-quality stack, of course.

Re: Docker, Mesos, Marathon, and the End of Pets

#24

Post by tomjen3 »

copsarebastards wrote: post #13

To be honest, I still have yet to see one of these systems that beats simply using Bash. They're all trying to make a scripting problem into a configuration problem. That's sometimes a reasonable idea, when what you're doing is common enough that only a few things here and there need to diverge from the defaults. But every image I've ever had to create contained far more edge cases than default cases, and half of tho…

In that case you might want to look at ansible: it does exactly what such a system should (logs in with SSH and runs some scripts) but does in a smart way (eg. you can configure certain boxes to be web servers and have it run a script on all web server boxes). It does have some weird config format but it does also allow you to run scripts.

Re: Docker, Mesos, Marathon, and the End of Pets

#25

Post by vezzy-fnord »

tomjen3 wrote: post #24
copsarebastards wrote: post #13

To be honest, I still have yet to see one of these systems that beats simply using Bash. They're all trying to make a scripting problem into a configuration problem. That's sometimes a reasonable idea, when what you're doing is common enough that only a few things here and there need to diverge from the defaults. But every image I've ever had to create contained far more edge cases than default cases, and half of tho…

In that case you might want to look at ansible: it does exactly what such a system should (logs in with SSH and runs some scripts) but does in a smart way (eg. you can configure certain boxes to be web servers and have it run a script on all web server boxes). It does have some weird config format but it does also allow you to run scripts.

Quite curious about if whether we had more robust shell languages, like Inferno's es, which uses functional expressions, though it's some 20+ years old by now, if kept alive by some enthusiasts... would much of the rationale behind the more basic configuration management and provisioning go away?

The chief rationale for embarking on the CM route is idempotence. Though it seems to me having a basic, lexer-only language simply for chain loading commands to exec() without much of the state and environment baggage behind full command shells could work as an alternative. execline works like this: http://skarnet.org/software/execline/

Re: Docker, Mesos, Marathon, and the End of Pets

#26

Post by jacques_chester »

qooleot wrote: post #14
jacques_chester wrote: post #5

Earlier quoted context omitted.

You might like Lattice[0], which is extracted from Cloud Foundry. Basically, everyone is racing back to PaaSes. Heroku pioneered it and are still out there. Red Hat have OpenShift and are making noises about turning it into a Docker+Kubernetes thing in version 3. Cloud Foundry has been around for a few years now. There are other also-rans. The thing is that apart from Heroku, you've not heard of installable PaaSes be…

"Red Hat have OpenShift and are making noises about turning it into a Docker+Kubernetes thing in version 3" They've already decided that, and have even reached code-freeze. Their conference is later this month, so that's when it's going to all be announced and rollout plans detailed. https://github.com/openshift/origin

I remember reading the design docs last year. Interesting times ahead.

Re: Docker, Mesos, Marathon, and the End of Pets

#27

Post by copsarebastards »

tomjen3 wrote: post #24
copsarebastards wrote: post #13

To be honest, I still have yet to see one of these systems that beats simply using Bash. They're all trying to make a scripting problem into a configuration problem. That's sometimes a reasonable idea, when what you're doing is common enough that only a few things here and there need to diverge from the defaults. But every image I've ever had to create contained far more edge cases than default cases, and half of tho…

In that case you might want to look at ansible: it does exactly what such a system should (logs in with SSH and runs some scripts) but does in a smart way (eg. you can configure certain boxes to be web servers and have it run a script on all web server boxes). It does have some weird config format but it does also allow you to run scripts.

I've used Ansible, and of the systems I've used to solve this and similar problems (Chef, Docker, Puppet) Ansible has been the cleanest, most well-thought-out, and easiest to use. But ultimately it's included in my review of the above tools. It misses 10% of cases instead of 50%, but I still end up spending an inordinate amount of time working around the system instead of running it.

Re: Docker, Mesos, Marathon, and the End of Pets

#28

Post by steveb »

bkeroack wrote: post #23
steveb wrote: post #12

The real problem is going from tutorial to something you would use in production. Throw in logging, security and service discovery and you can have a few engineers hacking away for months. So I want to plug a project I've been contributing to: https://github.com/CiscoCloud/microservices-infrastructure We're trying to make it super easy to deploy these tools. For example every time you launch a docker container, it wi…

This is basically why Kubernetes exists: for all the plumbing, discovery, etc required on top of bare containers. It still requires work to go from zero to production-quality stack, of course.

We like Kubernetes (and are looking to add it to our project), but our goal is to integrate building blocks that allows us to run many different types of workloads. Think of our project more like an Ubuntu for distributed systems.

Kubernetes may eventually spread out beyond Docker, but for today we need to support things like Kafka and Spark.

As others have noted, we've had things like CloudFoundry, OpenShift and Heroku, and these all-in-one frameworks tend not to extend outside their original domain.

Re: Docker, Mesos, Marathon, and the End of Pets

#29

Post by edgan »

sytse wrote: post #11
stephenr wrote: post #10

Earlier quoted context omitted.

While deb/rpm packages are better than a monolithic container, having used one of your rpms, I'd say they're only just barely better.

What didn't you like about the Omnibus rpms?

Omnibus packages are a horrible idea. They are security issues waiting to happen.

A great example is the Chef server rpm. It is a 500mb mini distribution in one package. It has copies of perl, python, Ruby, and Erlang in it. If any of these has a security vulnerability, I have to wait on the maintainer to release a new version, and hope it included the security fixes.

They also tend to include things like python header files for no reason. You wouldn't compile against an Omnibus package, but they are there anyway. Examples of this are Sumologic's and Datadog's agents.

Post reply on HN