Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mesosphere Announces First Data Center OS and $36M in Funding

gigaom.com

Post reply on HN
Print view View original

31–40 of 51 posts

Re: Mesosphere Announces First Data Center OS and $36M in Funding

#31

Post by justincormack »

dmpk2k wrote: post #28
presspot wrote: post #13

Earlier quoted context omitted.

There are a lot of components to an operating system. It's not just the technology components, it's the product components and the business components. E.g., Does it have an API? Does it have an SDK? Does it have a user interface? Does it have an init system, a chron, a storage system, service discovery? Does it have an ecosystem of third party developers? I posit that the OS Checklist is fairly long and that no of t…

I posit that the OS Checklist is fairly long and that no of the other systems you mention have the complete OS package. They do. E.g. SmartOS is a UNIX. It'd be pretty odd if it didn't have an init system . Or storage. Or cron. Or POSIX. Or whatever else you'd like to add to the list that UNIX systems usually have...

I assumed the OP meant a distributed init system, distributed storage, etc.

Re: Mesosphere Announces First Data Center OS and $36M in Funding

#32

Post by KaiserPro »

Older person here.

Whats nice is that people are thinking about supercomputers again, even if they insist on calling them "the cloud".

First things first, look up beowulf (http://en.wikipedia.org/wiki/Beowulf_cluster) which is a suite of tools that implents a multi machine scheduler, and message passing interface. Whats nice is that if one host is overloaded, it can migrate the process to another. (however I'm not sure what performance is like nowadays)

In the world of VFX for movies, we've been dealing with schedulers for years. Programmes like alfred, tractor, qube and deadline can dispatch tasks, and deal with dependencies at massive scale.

The first thing to note is that "DCOS" is really a discrete set of parts; a scheduler, machine state enforcement, network config, Storage, and the underlying OS.

With careful planning, the state enforcement tool (puppet and the like) can take care of all of these tasks except a global scheduler.

The beuty of the VFX scheduler is that they understand dependencies really well. (I need x to complete before I run y, I need feature z to run) A lot of newer schedulers really don't understand this concept well.

Re: Mesosphere Announces First Data Center OS and $36M in Funding

#33

Post by KaiserPro »

23david wrote: post #27
michaelsbradley wrote: post #23

Earlier quoted context omitted.

Nice tools, but (and it feels funny to say this) they seem rather "sticks and stones" in comparison to the integration achieved with open source PaaS layers like deis atop CoreOS; Mesosphere's DCOS even more so.

They're general purpose devops automation frameworks... you can choose to take any of them and make this 'deep integrated' layer you speak of. For example, Openstack deployments are commonly automated using Puppet scripts. (Really amazing stuff if you haven't seen it before...) DCOS may choose to reinvent the wheel, but unless there's a core innovation in the way they're handling orchestration or automation, I think…

Puppet and the like are state enforcement tools, not task schedulers.

As someone who used to run a 6000 core farm in 2007 (its not 25,000) I can tell you that puppet isn't going to help task placement. It can create machine that will run a certain app, but without some heavy programming it'll never balance or detect need and respond sensibly.

Re: Mesosphere Announces First Data Center OS and $36M in Funding

#34

Post by KaiserPro »

KaiserPro wrote: post #32

Older person here. Whats nice is that people are thinking about supercomputers again, even if they insist on calling them "the cloud". First things first, look up beowulf ( http://en.wikipedia.org/wiki/Beowulf_cluster ) which is a suite of tools that implents a multi machine scheduler, and message passing interface. Whats nice is that if one host is overloaded, it can migrate the process to another. (however I'm not…

Its really important to understand that puppet and the like cannot (without heavy engineering) act as a task dispatcher. The big feature of "DCOS" is task distribution.

In linux terms its like comparing the CPU scheduler to chmod. Yes you can make a program run by chmoding a file to +x, but its the scheduler that is responsible for making sure the programme has CPU time.

Re: Mesosphere Announces First Data Center OS and $36M in Funding

#35

Post by kelseyhightower »

Zariel wrote: post #30
kelseyhightower wrote: post #26

Disclosure: CoreOS employee; Kubernetes contributor. After reading the title I was a bit tempted to call BS since I think Kubernetes should have the rights to "The first Datacenter OS" tagline[1]. However, after reading up on the project details at https://mesosphere.com/learn I can see how Mesosphere came to the conclusion of being a DCOS, if not necessarily the first. Mesosphere goes a bit further than Kubernetes a…

Isn't it running ontop of the Mesos kernel, which has been around for longer than Kubernetes?

That was my first question as well, but there must be a reason why Kubernetes was brought in.

Re: Mesosphere Announces First Data Center OS and $36M in Funding

#36

Post by tomcart »

DCOS is an interesting description, as the idea of a data centre (to my tiny mind at least) is made more fuzzy by concepts like AWS AZs.

Do people expect that the 'DC' will span AZs, regions even? Or is the separation of these things valuable in some way?

How about the idea of dev vs prod environments? Will the isolation provided be strong enough that we'll happily drop everything onto a single cluster of machines?

Re: Mesosphere Announces First Data Center OS and $36M in Funding

#37

Post by presspot »

Zariel wrote: post #30
kelseyhightower wrote: post #26

Disclosure: CoreOS employee; Kubernetes contributor. After reading the title I was a bit tempted to call BS since I think Kubernetes should have the rights to "The first Datacenter OS" tagline[1]. However, after reading up on the project details at https://mesosphere.com/learn I can see how Mesosphere came to the conclusion of being a DCOS, if not necessarily the first. Mesosphere goes a bit further than Kubernetes a…

Isn't it running ontop of the Mesos kernel, which has been around for longer than Kubernetes?

A bit of the history:

The Mesosphere DCOS is built around the Apache Mesos kernel

The Mesos kernel was developed at UC Berkeley in 2009 [1].

Spark was written as a sample app on top of it [2].

Ben Hindman and his colleagues at the UC Berkeley AmpLab had always envisioned Mesos as a kernel inside of a full-blown operating system [3]. They finally brought it to market.

[1] https://www.usenix.org/legacy/event/nsdi11/tech/full_papers/...

[2] "We have implemented Mesos in 10,000 lines of C++. The system scales to 50,000 (emulated) nodes and uses ZooKeeper for fault tolerance. To evaluate Mesos, we have ported three cluster computing systems to run over it: Hadoop, MPI, and the Torque batch scheduler. To validate our hypothesis that specialized frameworks provide value over general ones, we have also built a new framework on top of Mesos called Spark, optimized for iterative jobs where a dataset is reused in many parallel operations, and shown that Spark can outperform Hadoop by 10x in iterative machine learning workloads." ibid.

[3] http://people.csail.mit.edu/matei/papers/2011/hotcloud_datac...

Re: Mesosphere Announces First Data Center OS and $36M in Funding

#38

Post by presspot »

presspot wrote: post #37
Zariel wrote: post #30

Earlier quoted context omitted.

Isn't it running ontop of the Mesos kernel, which has been around for longer than Kubernetes?

A bit of the history: The Mesosphere DCOS is built around the Apache Mesos kernel The Mesos kernel was developed at UC Berkeley in 2009 [1]. Spark was written as a sample app on top of it [2]. Ben Hindman and his colleagues at the UC Berkeley AmpLab had always envisioned Mesos as a kernel inside of a full-blown operating system [3]. They finally brought it to market. [1] https://www.usenix.org/legacy/event/nsdi11/tec…

So, yes, Mesos predates Kubernetes by many years.

Re: Mesosphere Announces First Data Center OS and $36M in Funding

#39

Post by presspot »

kelseyhightower wrote: post #35
Zariel wrote: post #30

Earlier quoted context omitted.

Isn't it running ontop of the Mesos kernel, which has been around for longer than Kubernetes?

That was my first question as well, but there must be a reason why Kubernetes was brought in.

I see Kubernetes as more of a programming model, not an operating system. It provides a way to express services and have them scheduled onto a datacenter. The rocket-science of how you schedule those workloads and optimize them in the same partition as other workloads is what you need a DCOS for.

In the Mesosphere world, Kubernetes is a "datacenter service" which is installed on your datacenter so that you can run Kubernetes workloads. You might also want to install DEIS to run DEIS-organized workloads. Or Spark, for Spark workloads... and so on -- all multitenant in the cluster. This is what the DCOS is uniquely good at, and why it qualifies as a true operating system.

Re: Mesosphere Announces First Data Center OS and $36M in Funding

#40

Post by fsaintjacques »

kelseyhightower wrote: post #26

Disclosure: CoreOS employee; Kubernetes contributor. After reading the title I was a bit tempted to call BS since I think Kubernetes should have the rights to "The first Datacenter OS" tagline[1]. However, after reading up on the project details at https://mesosphere.com/learn I can see how Mesosphere came to the conclusion of being a DCOS, if not necessarily the first. Mesosphere goes a bit further than Kubernetes a…

I didn't see any reference to storage solution, am I misreading?
Post reply on HN