Google Launches Managed Service for Running Docker Apps on Its Platform

Kubernetes and GKE operate at a different level. It is an API for being able to schedule and manage containers instead of VMs. At this point, k8s/GKE doesn't have an idea of an "app" any more than a VM IaaS service has an idea of an app.

Moving from a VM centric view to a container centric view improves a lot of things: * Easier to create and manage images * Portability -- images can be moved between providers. Develop on your laptop with docker/k8s and deploy the exact same container image. * More transparency into compute workload. The hosting VM or cloud can see more of what is going on in the container for monitoring and logging, etc. * More efficient/higher density -- you can run more on any piece of (virtual?) hardware * More flexible resource sharing/overcommit -- as you drive density up you can get more nuanced about what workloads take get priority.

Some of this stuff is still in its infancy -- complex resource models aren't fully supported in Docker yet -- but it is where things are going, at least based on Google's experience.

Spark is a successor to Hadoop, aiming to perform big distributed data crunching jobs more quickly by not limiting itself to a map-reduce paradigm and by holding more data in memory.

Apache Mesos is a resource scheduler for a cluster of machines. It is architected to be agnostic about the kind of application running on it, but I get the impression it's primarily used to make things like Hadoop and Spark run on the same cluster without stepping on each other's toes (and not used so much for long-running services like a web app). An application says to Mesos "hey I'd like to run job X", outlining the resources that it needs, and then Mesos looks at the cluster to see the best place to run the job. Mesos has some Docker integration but it's weird. Instead of launching your job and creating cgroups directly as it usually does, Mesos has to run a proxy process that talks to Docker and asks it to do it. (This is because Docker no longer has standalone mode. Which is my biggest gripe about Docker.)

Kubernetes is solving a similar problem to Mesos, how to allocate resources in a cluster to all the jobs that you might want to run on it. It's explicitly focused on "containers", though not necessarily just Docker containers. (The README at https://github.com/GoogleCloudPlatform/kubernetes makes no mention of Docker.) I get the impression it's focused more on long-running processes like web servers and less on ephemeral Hadoop-style jobs.