Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run Kubernetes on top of DC/OS

mesosphere.com

Post reply on HN
Print view View original

31–37 of 37 posts

Re: Run Kubernetes on top of DC/OS

#31

Post by bogomipz »

tknaup wrote: post #20
bogomipz wrote: post #12

>"Kubernetes on top of Mesos through DC/OS more closely matches Google’s own architecture; where Kubernetes is a service running within VMs that are managed by Google’s proprietary Borg platform." My understanding was that Google runs containers in a VM for security. Mesos uses the Docker container executor not a VM. How does this more closely resemble Google's Borg/VM model?

Mesosphere co-founder here. You're correct in that GCP runs k8s in VMs, DC/OS doesn't. What's similar is that there's a resource manager underneath - Borg for GCP, Mesos for DC/OS. They serve similar purposes like resource management, isolation, and operating the services on top.

>"What's similar is that there's a resource manager underneath - Borg for GCP, Mesos for DC/OS."

Maybe I don't fully understand DC/OS then. I was under the impression that DC/OS was simply a distro for Mesosphere. But your comment make me think that either my understanding is incorrect or else DC/OS has become something more than a Mesos distro. Could you elaborate? Thanks.

Re: Run Kubernetes on top of DC/OS

#32

Post by bogomipz »

estill01 wrote: post #26
bogomipz wrote: post #21

Earlier quoted context omitted.

Thanks, can you elaborate on the libs available to "migrate your data around"? Does Mesosphere reschedule your DB to another node that has an equivalent persistent and reserved storage(SSD etc.) volume configured on it? Are you using Portworx for this?

> "can you elaborate on the libs available to 'migrate your data around'? ... Are you using Portworx for this?" The tool I had in mind was 'REX-Ray'[ https://mesosphere.github.io/marathon/docs/external-volumes.... ]. That said, we're not actually doing the 'chasing db' config. Instead we run a HA Neo4j DB deployment as a Marathon service pegged to a handful of nodes each with local persistent volumes allocated to Neo…

Thanks for the explanation. I've heard of Rex-RAY but I thought that was a vendor-specific solution(EMC.) Maybe that has changed?

>"We'll probably opt for this config near-term for our web-server SSR cache."

What is an SSR cache?

Re: Run Kubernetes on top of DC/OS

#33

Post by bogomipz »

tknaup wrote: post #13
bogomipz wrote: post #10

I was confused by this passage: >"Running Kubernetes on DC/OS allows you to run different types of workloads (more explicitly, both the stateless and stateful components that make up most modern applications) on the same infrastructure." Can someoene answer - how does Running Kubernetes on top of DC/OS help you run stateful apps on Kubernetes? Or is the meaning that DC/OS is better for running stateful services and t…

Mesosphere co-founder here. This is correct: "DC/OS is better for running stateful services and then you can use K8 to run your stateless services" Data services run directly on DC/OS via application-aware schedulers. They have the operational logic for how to bring up say a Cassandra cluster correctly, how to upgrade it to a new version without breaking it, change config, scale up, etc. All things you usually have t…

Thanks for the clarification.

Re: Run Kubernetes on top of DC/OS

#34

Post by jdc0589 »

jacques_chester wrote: post #16
slap_shot wrote: post #3

I recently reached out to Mesosphere to better understand the value and options available to my company if we switched to Mesos (and thus Mesosphere). We currently run Kubernetes across all major cloud providers and we'd like some help. I was startled by how dogmatic they are about their pricing model: per node, annually. Our entire business model is running dynamically scaling data pipelines for companies. We spin n…

Disclosure: I work for Pivotal, which competes with Mesosphere in this arena. Read with discretion. If shilling persists, consult your techcrunch. With that out of the way: you should try Kubo. Google & Pivotal have been working on Kubo[0] to make the management of Kubernetes easier by using BOSH as the deployment/update/repair system. You make deployments by editing a yaml file. Or, better yet, but having a tool edi…

I'm gonna check kubo out, cause its REALLY relevant to me in the coming months. Can you comment any why kubo might be preferable to something like kops?

You are probably aware of this, but people are starting to get a little fatigued from amount of automated ops/deploy tooling out there for k8s. Don't get me wrong, this is an area that needs improvement so competition is totally necessary, but its starting to become a bit difficult to navigate the landscape, or even keep up with the best choices.

Re: Run Kubernetes on top of DC/OS

#35

Post by jacques_chester »

jdc0589 wrote: post #34
jacques_chester wrote: post #16

Earlier quoted context omitted.

Disclosure: I work for Pivotal, which competes with Mesosphere in this arena. Read with discretion. If shilling persists, consult your techcrunch. With that out of the way: you should try Kubo. Google & Pivotal have been working on Kubo[0] to make the management of Kubernetes easier by using BOSH as the deployment/update/repair system. You make deployments by editing a yaml file. Or, better yet, but having a tool edi…

I'm gonna check kubo out, cause its REALLY relevant to me in the coming months. Can you comment any why kubo might be preferable to something like kops? You are probably aware of this, but people are starting to get a little fatigued from amount of automated ops/deploy tooling out there for k8s. Don't get me wrong, this is an area that needs improvement so competition is totally necessary, but its starting to become…

> Can you comment any why kubo might be preferable to something like kops?

I'm unqualified to give a fair comparison, as I'm only skim-the-website familiar with kops. BOSH comes up much more frequently in my work. Most teams working on Cloud Foundry use it in some way, even if only to manage Concourse.

The main advantage that BOSH has over any of the others is maturity and production experience with large, stateful, distributed systems (first release was in 2010). It got the original abstractions right in a way that the alternatives of the time didn't. Chef et al are pet-builders, they excel in wrangling a single server into the target state you want.

BOSH instead says: why do you care about single servers? You're building a distributed system. If components drift or break, replace them with a clean image which was built from source.

As an example of production use, at Pivotal we use it to manage PWS. The Cloudops team use BOSH to roll out changes to a system running 10s of thousands of apps from thousands of users and companies.

In general, unless we land on a bug in the code rolled out, nobody ever notices. When we do have a bug, we can roll it back pretty easily.

BOSH isn't constrained to deploying Kubernetes. It was originally developed for Cloud Foundry and since then people have packaged up all manner of systems for it. We support some ourselves. For example, we have releases for RabbitMQ, MySQL and so on.

These can get used by on-demand service brokers too. Say an app developer wants a private RabbitMQ cluster. They tell the service broker to create a service, it has BOSH setup and monitor a brand new cluster, when it's done the dev can bind it to their app with a single command. Bing bang boom, totally self-service services. Nobody needs to fill out a risk form, file a ticket or pester their inside connection in ops.

One last advantage for Kubo and BOSH generally is that Google has assigned fulltime Googlers to both of them in multiple locations, working in pairs alongside Pivots. We've also become closely engaged with Google's new CRE program and it's been a really great learning experience for us.

Re: Run Kubernetes on top of DC/OS

#36

Post by afulay »

bogomipz wrote: post #32
estill01 wrote: post #26

Earlier quoted context omitted.

> "can you elaborate on the libs available to 'migrate your data around'? ... Are you using Portworx for this?" The tool I had in mind was 'REX-Ray'[ https://mesosphere.github.io/marathon/docs/external-volumes.... ]. That said, we're not actually doing the 'chasing db' config. Instead we run a HA Neo4j DB deployment as a Marathon service pegged to a handful of nodes each with local persistent volumes allocated to Neo…

Thanks for the explanation. I've heard of Rex-RAY but I thought that was a vendor-specific solution(EMC.) Maybe that has changed? >"We'll probably opt for this config near-term for our web-server SSR cache." What is an SSR cache?

Yes, Rex-Ray is EMC specific. Take a look at Robin Systems (https://robinsystems.com/) for stateful containers. They have examples of running hadoop, cassandra, mongodb, etc all on commodity hardware.

Re: Run Kubernetes on top of DC/OS

#37

Post by tobilg »

afulay wrote: post #36
bogomipz wrote: post #32

Earlier quoted context omitted.

Thanks for the explanation. I've heard of Rex-RAY but I thought that was a vendor-specific solution(EMC.) Maybe that has changed? >"We'll probably opt for this config near-term for our web-server SSR cache." What is an SSR cache?

Yes, Rex-Ray is EMC specific. Take a look at Robin Systems ( https://robinsystems.com/ ) for stateful containers. They have examples of running hadoop, cassandra, mongodb, etc all on commodity hardware.

That's not really accurate I think. RexRay is also supporting non-EMC solutions: https://github.com/codedellemc/rexray#storage-provider-suppo...
Post reply on HN