Kubeflow – Machine Learning Toolkit for Kubernetes
1–10 of 35 posts
Re: Kubeflow – Machine Learning Toolkit for Kubernetes
#2This looks pretty cool. Is it dependent on Google’s kubernetes or can it be run on Openshift or DC/OS as well?
Re: Kubeflow – Machine Learning Toolkit for Kubernetes
#3Looks like different components could be added in the future, but not clear how.
The following are included:
- A JupyterHub to create & manage interactive Jupyter notebooks.
- A Tensorflow Training Controller that can be configured to use CPUs or GPUs, and adjusted to the size of a cluster with a single setting.
- A TF Serving container.
Re: Kubeflow – Machine Learning Toolkit for Kubernetes
#4This looks pretty cool. Is it dependent on Google’s kubernetes or can it be run on Openshift or DC/OS as well?
It says it runs in "in any environment in which Kubernetes runs." So as long as you are asking if it runs on Openshift's Kubernetes, than yes.
Re: Kubeflow – Machine Learning Toolkit for Kubernetes
#5Pretty interesting. I'm guessing this is something that Google uses internally for their Kubernetes workflows.
Re: Kubeflow – Machine Learning Toolkit for Kubernetes
#6Pretty interesting. I'm guessing this is something that Google uses internally for their Kubernetes workflows.
I am at KubeCon 2017 in Austin, TX and, yeah, based on the presentation, it looks like an internal tool they just opened to the public with some bold goals.
Re: Kubeflow – Machine Learning Toolkit for Kubernetes
#7Hi! I’m David Aronchick, PM on Kubeflow, I’m happy to answer any questions! I was one of the early PMs on Kubernetes, and we very much want to make this a community project, so please join us in thinking about what’s next!
- GH: https://GitHub.com/Google/Kubeflow
- kubeflow-Discuss: https://groups.google.com/forum/m/#!forum/kubeflow-discuss
NOTE: The name "flow" does not refer directly to TensorFlow; if anything it's a nod at all the river themes that pop up in the ML community (eg FBLearner Flow)
Disclosure: I work at Google on Kubeflow
Re: Kubeflow – Machine Learning Toolkit for Kubernetes
#8Looks like different components could be added in the future, but not clear how. The following are included: - A JupyterHub to create & manage interactive Jupyter notebooks. - A Tensorflow Training Controller that can be configured to use CPUs or GPUs, and adjusted to the size of a cluster with a single setting. - A TF Serving container.
Correct! We're currently thinking a lot about orchestration of the various components but for now, our goal is to use the native loose coupling between services available in K8s. So if you wanted Spark for data processing, for example, you could start a service, and the deployment, and feed that into the TF CRD.
Disclosure: I work at Google
Re: Kubeflow – Machine Learning Toolkit for Kubernetes
#9This looks pretty cool. Is it dependent on Google’s kubernetes or can it be run on Openshift or DC/OS as well?
The controller part is a Custom Resource Definition. It can run on any cloud.
However, to benefit from GPUs you need to configure the controller correctly. Default configuration are there for GCP and Azure, but you would need to do that manually for other cloud (not that it is very hard)
Re: Kubeflow – Machine Learning Toolkit for Kubernetes
#10Pretty interesting. I'm guessing this is something that Google uses internally for their Kubernetes workflows.
Its very close to how we think about ML internally, but not what we use. Your best bet to read that is look at the TFX paper[1] which describes our internal thoughts in great detail. (Though Kubeflow is not designed to be an externalization of TFX, we're very much working in collaboration with that team)
[1] http://www.kdd.org/kdd2017/papers/view/tfx-a-tensorflow-base...
Disclosure: I work at Google on Kubeflow