A PyTorch Approach to ML Infrastructure

Looks like Runhouse is FOSS (Apache 2.0) and you get to choose your own infrastructure. I will try out Runhouse. Mojo wants me to send them my info to get started.

Very interesting. I just worked to implement a baby version of this kind of system at work. Similar to this project, our basic use case was allowing researchers to quickly/easily execute their arbitrary R&D code on cloud resources. It's difficult to know in advance what they might be doing, and we wanted to avoid a situation where they are pushing a docker container or submitting a file every time they change something. So we made it possible for them to "just" ship a single class/function without leaving their local interactive environment.

I see from looking at the source here, run.house is using the same approach of cloudpickling the function. That works, but one struggle we are having is it's quite brittle. It's all gravy assuming everyone is operating in perfectly fresh environments that mirror the cluster, but this is rarely the case. Even subtle changes in the execution environment locally can produce segfaults when run on the server. Very hard to debug. The code here looks a lot more mature, so I'm assuming this is more robust than what we have. But would be curious if the developers have run into similar challenges.

This is a cool approach. I really like the notion of small, powerful components that compose well together. ML infra is sorely missing this piece. I wish you the best of luck!

That's a good question. I actually love the Mojo concept, but see it as very different. They're creating a portable acceleration option in Python proper, while we're trying to make it so you can easily ship around such code to different infra. You can see them or other DSLs like Ray as handling "inside the cluster" while we're focused on solving "outside the cluster." That's what I've picked up from their marketing but could be missing something.

In general making code itself more portable is great (which is the objective of many ML compilers) and will make Runhouse even more valuable, because the ability to take the same code and send it to different places shines when those different places can be different compute types.