Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run Kubernetes on top of DC/OS

mesosphere.com

Post reply on HN
Print view View original

11–20 of 37 posts

Re: Run Kubernetes on top of DC/OS

#11

Post by rectang »

don71 wrote: post #9
rectang wrote: post #7

Earlier quoted context omitted.

> Mesos (and thus Mesosphere) Mesos is an open source project. Would you consider any other vendors besides Mesosphere?

DC/OS is also open source. What are your concerns with it?

Mesos is an Apache project. (DC/OS is not.) Apache projects are required to be vendor-neutral. That's part of the covenant under which contributors participate.

The Apache governance model provides opportunities for competing vendors, so besides Mesosphere, the market is open for other players to provide Mesos support.

Re: Run Kubernetes on top of DC/OS

#12

Post by bogomipz »

>"Kubernetes on top of Mesos through DC/OS more closely matches Google’s own architecture; where Kubernetes is a service running within VMs that are managed by Google’s proprietary Borg platform."

My understanding was that Google runs containers in a VM for security. Mesos uses the Docker container executor not a VM. How does this more closely resemble Google's Borg/VM model?

Re: Run Kubernetes on top of DC/OS

#13

Post by tknaup »

bogomipz wrote: post #10

I was confused by this passage: >"Running Kubernetes on DC/OS allows you to run different types of workloads (more explicitly, both the stateless and stateful components that make up most modern applications) on the same infrastructure." Can someoene answer - how does Running Kubernetes on top of DC/OS help you run stateful apps on Kubernetes? Or is the meaning that DC/OS is better for running stateful services and t…

Mesosphere co-founder here.

This is correct: "DC/OS is better for running stateful services and then you can use K8 to run your stateless services"

Data services run directly on DC/OS via application-aware schedulers. They have the operational logic for how to bring up say a Cassandra cluster correctly, how to upgrade it to a new version without breaking it, change config, scale up, etc. All things you usually have to figure out yourself. When you run k8s on DC/OS you get these same benefits.

Re: Run Kubernetes on top of DC/OS

#14

Post by wmf »

slap_shot wrote: post #3

I recently reached out to Mesosphere to better understand the value and options available to my company if we switched to Mesos (and thus Mesosphere). We currently run Kubernetes across all major cloud providers and we'd like some help. I was startled by how dogmatic they are about their pricing model: per node, annually. Our entire business model is running dynamically scaling data pipelines for companies. We spin n…

This makes sense if DC/OS is targeted exclusively at on-prem.

Re: Run Kubernetes on top of DC/OS

#15

Post by estill01 »

bogomipz wrote: post #10

I was confused by this passage: >"Running Kubernetes on DC/OS allows you to run different types of workloads (more explicitly, both the stateless and stateful components that make up most modern applications) on the same infrastructure." Can someoene answer - how does Running Kubernetes on top of DC/OS help you run stateful apps on Kubernetes? Or is the meaning that DC/OS is better for running stateful services and t…

Currently running stateful apps on a DC/OS cluster + HA DBs. Fairly straight forward to get stateful working now + there are libs to migrate data around to chase your apps/services when/if Mesos/Marathon relocates them (e.g. after a service restart/crash) if you need your data co-located with a managed app/service.

The thing I'm most interested in exploring with K8+DC/OS is having DC/OS manage a couple of K8 instances so we can isolate 'virtual' clusters for various envs/apps. I suppose you could do this already with Marathon (the DC/OS built-in container manager), but we're not. Psyched to benefit from the K8 community + have the underlying DC/OS VM control/management plane.

Re: Run Kubernetes on top of DC/OS

#16

Post by jacques_chester »

slap_shot wrote: post #3

I recently reached out to Mesosphere to better understand the value and options available to my company if we switched to Mesos (and thus Mesosphere). We currently run Kubernetes across all major cloud providers and we'd like some help. I was startled by how dogmatic they are about their pricing model: per node, annually. Our entire business model is running dynamically scaling data pipelines for companies. We spin n…

Disclosure: I work for Pivotal, which competes with Mesosphere in this arena. Read with discretion. If shilling persists, consult your techcrunch.

With that out of the way: you should try Kubo.

Google & Pivotal have been working on Kubo[0] to make the management of Kubernetes easier by using BOSH as the deployment/update/repair system. You make deployments by editing a yaml file. Or, better yet, but having a tool edit the file for you as part of a structured CI/CD pipeline (I've seen both and I am a fan of the latter).

BOSH works at the IaaS layer. Insofar as you mean "spin up and down VMs", Kubo is well-suited for that exact problem. You also get a bunch of upgrade and self-healing stuff for free.

Pivotal is also adding a commercial offering based on Kubo, PKS[1], which has been built with Google[2] and VMWare[3]. You might not need it for your exact usecase; the sweetspot is Pivotal Cloud Foundry users who want a relatively seamless side-by-side integration between CF and k8s. There's a bunch of value-added features (Harbor, GCP service brokers), of course, plus the whole throat-to-choke thing people like to pay for.

The pricing model isn't set in stone, but I suspect it will settle in the vicinity of what you have in mind. Our DNA is charging per-instance for Cloud Foundry apps, rather than worrying about sockets or RAM or other metrics which only poorly correlate to user value.

Anyway, I can hook you up with Kubo or PKS folk. Email me: jchester@pivotal.io

[0] https://github.com/cloudfoundry-incubator/kubo-deployment

[1] https://pivotal.io/pks

[2] https://www.blog.google/topics/google-cloud/vmware-and-pivot...

[3] https://blogs.vmware.com/cloudnative/2017/08/29/vmware-pivot...

Re: Run Kubernetes on top of DC/OS

#17

Post by devrandomguy »

Could anyone point out the boundary between the open-source Mesos / Marathon stack, and the commercial DC/OS? The official documentation (https://docs.mesosphere.com/1.9/overview/what-is-dcos/) doesn't make it clear. What capabilities does DC/OS have that are not available in the open-source portion of the stack?

> As a distributed system, DC/OS includes a group of agent nodes that are coordinated by a group of master nodes. Like other distributed systems, several of the components running on the master nodes perform leader election with their peers.

This just sounds like vanilla Mesos masters and slaves. DC/OS "runs on top of" that, but what is it actually doing? Is the DC/OS another service that just enables the Mesos masters to initially discover and replace each other? Could it not be replaced with Zookeeper or Consul? That seems like a small piece of the puzzle, what would make it the one expensive thing, while the rest of the system is free? Is the overall stack actually shareware, rather than a community of independent open-source services?

The reason I am so curious, is because after running a small demo a year ago, the Mesos stack looked really promising. The only thing holding me back from proposing a large scale trial, for comparison to our sprawling Heroku/ECS/EBS setup, was the feeling that I was not understanding a crucial part of the architecture, and not understanding the pricing, if that is even applicable (couldn't find price info anywhere! How do I quantify that part?)

Re: Run Kubernetes on top of DC/OS

#18

Post by trengrj »

don71 wrote: post #9
rectang wrote: post #7

Earlier quoted context omitted.

> Mesos (and thus Mesosphere) Mesos is an open source project. Would you consider any other vendors besides Mesosphere?

DC/OS is also open source. What are your concerns with it?

Components aren't open source. Still waiting for the new hdfs package to be released see https://github.com/mesosphere/hdfs-deprecated/blob/master/RE...

Re: Run Kubernetes on top of DC/OS

#19

Post by pivotalpoint415 »

CoffeeDregs wrote: post #5
slap_shot wrote: post #3

I recently reached out to Mesosphere to better understand the value and options available to my company if we switched to Mesos (and thus Mesosphere). We currently run Kubernetes across all major cloud providers and we'd like some help. I was startled by how dogmatic they are about their pricing model: per node, annually. Our entire business model is running dynamically scaling data pipelines for companies. We spin n…

Had an interesting pricing experience with them a couple of years ago. We were happy with our Mesos/Marathon cluster of ~100-200 machines but needed some acute support and we were in the same investment fund of DCVC. Got on the phone with ... Sales... No problem; you're a friend of ours because of DCVC; we'll get you special, friendly pricing on DC/OS... Got an email a few hours late: $100,000 per year. Was fairly pu…

A lot of software vendors charge $X,000 / server / year in subscription fees.

Redhat charges > $10,000 per physical server for their OpenShift and Pivotal gets for 100s of nodes often 7-figures. Remember just how much VMWare costs...

Re: Run Kubernetes on top of DC/OS

#20

Post by tknaup »

bogomipz wrote: post #12

>"Kubernetes on top of Mesos through DC/OS more closely matches Google’s own architecture; where Kubernetes is a service running within VMs that are managed by Google’s proprietary Borg platform." My understanding was that Google runs containers in a VM for security. Mesos uses the Docker container executor not a VM. How does this more closely resemble Google's Borg/VM model?

Mesosphere co-founder here.

You're correct in that GCP runs k8s in VMs, DC/OS doesn't. What's similar is that there's a resource manager underneath - Borg for GCP, Mesos for DC/OS. They serve similar purposes like resource management, isolation, and operating the services on top.

Post reply on HN