Kubeflow – Machine Learning Toolkit for Kubernetes • Hacker News Board

Re: Kubeflow – Machine Learning Toolkit for Kubernetes

#31

Post by ethbro » Fri, Dec 08, 2017, 2:54 PM UTC

TheIronYuppie wrote: post #24

ecnahc515 wrote: post #20

Earlier quoted context omitted.

Do you foresee some kind of integration between Google Cloud ML engine and Tensorflow on k8s in the future?

We're always ready to talk roadmap - anything in particular you'd like to see integration-wise? Disclosure: I work at Google on Kubeflow

Off the top of my head, maybe a maintained "ml-engine aligned" kubeflow setup, to the extent that's possible.

The use case I'm think of is an ml dev team building on kubeflow and proving a system. Then wanting to transfer it to a non-engineering team, yet wash their hands of any ongoing infrastructure ops responsibility.

Knowing that a "ml-engine aligned" kubeflow config would transfer cleanly (including associated bells and whistles) would make that a much more attractive option.

Caveat: I'll admit I'm not keeping up on what's in the managed offering, but I'm assuming there are a number of value-adds of the type that end users like (visualizations, etc).

Re: Kubeflow – Machine Learning Toolkit for Kubernetes

#32

Post by TheIronYuppie » Fri, Dec 08, 2017, 5:20 PM UTC

ethbro wrote: post #31

TheIronYuppie wrote: post #24

Earlier quoted context omitted.

We're always ready to talk roadmap - anything in particular you'd like to see integration-wise? Disclosure: I work at Google on Kubeflow

Off the top of my head, maybe a maintained "ml-engine aligned" kubeflow setup, to the extent that's possible. The use case I'm think of is an ml dev team building on kubeflow and proving a system. Then wanting to transfer it to a non-engineering team, yet wash their hands of any ongoing infrastructure ops responsibility. Knowing that a "ml-engine aligned" kubeflow config would transfer cleanly (including associated b…

Yes, this is EXACTLY what we're trying to do! However, it's a bit early, so I can't say when or where we'll be able to get to it. Also, I should be clear, though I'm from Google, we would really like the same story to work with other cloud's hosted offerings as well, but we'll need their support to do so!

Disclosure: I work at Google on Kubeflow

Re: Kubeflow – Machine Learning Toolkit for Kubernetes

#33

Post by TheIronYuppie » Fri, Dec 08, 2017, 5:23 PM UTC

bonesss wrote: post #30

minerals wrote: post #29

Earlier quoted context omitted.

Interesting, though I don't see how it is better than a plain docker image over kubernetes? Not much of a hassle now too. And how is it different from what DL4J is already doing with Zeppelin and supporting both Keras, TF, MXNet and PyTorch on the way?

> ...how [is it] better than a plain docker image over kubernetes? Scalability for people with existing on-premise (or cloud based), kubernetes workflows, especially once it comes to training or heavy crunching. That's not to say that Docker Machine/Swarm/Compose couldn't handle the same, but it's an extra step for kubernetes users and pushes people onto a slightly different toolchain than minikube->K8s.

Correct! Many folks have more complicated deployments in the cloud, and we're trying to align (as close as humanly possible) your on-prem stack with your cloud stack, to minimize the pain in migration.

If you have a single container, and a simple pipeline, this may be a bit more than you need. We've just found that there are normally 5 or more services/systems that people wire together to create an ML stack, and that's what we're trying to solve for/simplify.

Disclosure: I work at Google on Kubeflow

Re: Kubeflow – Machine Learning Toolkit for Kubernetes

#34

Post by TheIronYuppie » Fri, Dec 08, 2017, 5:23 PM UTC

quadrature wrote: post #25

TheIronYuppie wrote: post #22

Earlier quoted context omitted.

We're absolutely looking at it! Please join our discussion, we'd love to talk about what you're building and if we can help and/or what you'd like us to OSS. Disclosure: I work at Google on Kubeflow

Sure, is there an issue/doc/pr to comment on ?.

No, would you mind adding one?

Re: Kubeflow – Machine Learning Toolkit for Kubernetes

#35

Post by sub-mod » Mon, Dec 11, 2017, 3:58 AM UTC

smooc wrote: post #2

This looks pretty cool. Is it dependent on Google’s kubernetes or can it be run on Openshift or DC/OS as well?

With Openshift you can also exploit the S2I feature(This is not based on CRD.)You can refer this Blog on some ways which this can be done for TF. https://blog.openshift.com/openshift-commons-briefing-110-co...