I wonder why UC Berkeley doesn't build a proper HPC, they have a Data School and should provide this service for free to their faculties. We have "free" HPC resources at TU Dresden (Germany) (meaning: faculties do not need to pay for using HPC resources and they are not calculated in project budgets). I once applied for a job at University of Virginia, and they didn't have a HPC - everything was bought from AWS. When…
UC Berkeley launches SkyPilot to help navigate soaring cloud costs
21–30 of 114 posts
Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs
#22I feel that the old SETI@home project can use a comeback. Most home machines have powerful GPU these days. The ML problems are embarrassingly parallel and can be distributed to the home machines. Just need to work out the economy for everyone involved.
Datacenter GPUs have >1000gbps network connections between nodes, which is necessary to actually utilize GPUs with current training techniques. It's possible that a furthering of techniques used in GPT-JT[1] might make it feasible to use home computers, but even GPT-JT requires at least a 1gbps connection.
[1]: https://www.together.xyz/blog/releasing-v1-of-gpt-jt-powered...
Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs
#23At some point people will rediscover that "buying" whole sets of equipment through a lease financing company on a lease-to-own plan with $1 end payment, and colocating it the traditional way can often be significantly cheaper than paying endless "cloud" costs. naysayers will say, but the cloud allows you abstract away your salary costs of engineers! look at all the people you aren't hiring! I say: If your needs are s…
[deleted]
Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs
#24Earlier quoted context omitted.
They have a project which addresses this concern as well: https://skyplane.org/en/latest/benchmark.html
I'm one of the creators of Skyplane. Skyplane can migrate large datasets between cloud regions at 10s of Gbps while compressing data to reduce egress fees. Happy to chime in! https://github.com/skyplane-project/skyplane
Congrats on the launch! I had a similar idea once a few years back but failed to materialize it. You might want to consider other cloud providers like Sushi Cloud to get costs even lower. Happy to do an intro if it seems interesting.
Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs
#25In my experience, if you want _the_ best price-performance ratio for ML jobs, your best bet is CoreWeave. They have great GPU availability and it's a lot easier to scale up/down very quickly than on the popular clouds. I would really be surprised if anyone else out there is offering a better price point for these types of workloads..
Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs
#26I feel that the old SETI@home project can use a comeback. Most home machines have powerful GPU these days. The ML problems are embarrassingly parallel and can be distributed to the home machines. Just need to work out the economy for everyone involved.
Single ML job can’t be distributed very well. More often than not they hit network limit, even on single zone 1 GB/s network speed that we normally get. Most of the distributed workloads use something like NVLink.
That's true. Most ML algorithms have the iterate-until-converge pattern. How about some tasks like hyper parameter tuning or trying out different algorithms against the same data set? Those can be run in parallel.
ML research should aim to produce more parallel algorithms.
Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs
#27This is great for list price workloads. But once you have large usage (N+ MM/year) you’d likely get better savings by doing a multi-year usage contract.
Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs
#28Earlier quoted context omitted.
I'm one of the creators of Skyplane. Skyplane can migrate large datasets between cloud regions at 10s of Gbps while compressing data to reduce egress fees. Happy to chime in! https://github.com/skyplane-project/skyplane
Congrats on the launch! I had a similar idea once a few years back but failed to materialize it. You might want to consider other cloud providers like Sushi Cloud to get costs even lower. Happy to do an intro if it seems interesting.
Or to leverage cheaper compute/energy when it’s available. https://www.crusoecloud.com/features/
Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs
#29In my experience, if you want _the_ best price-performance ratio for ML jobs, your best bet is CoreWeave. They have great GPU availability and it's a lot easier to scale up/down very quickly than on the popular clouds. I would really be surprised if anyone else out there is offering a better price point for these types of workloads..
checkout lambda labs, they have very competitive pricing on GPU, they are almost half the price of CoreWeave at the last check.
Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs
#30In my experience, if you want _the_ best price-performance ratio for ML jobs, your best bet is CoreWeave. They have great GPU availability and it's a lot easier to scale up/down very quickly than on the popular clouds. I would really be surprised if anyone else out there is offering a better price point for these types of workloads..
checkout lambda labs, they have very competitive pricing on GPU, they are almost half the price of CoreWeave at the last check.
Indeed! Having lower-cost GPU clouds in the "Sky" is on our immediate roadmap: https://github.com/skypilot-org/skypilot/blob/master/ROADMAP...
In fact, as we speak we're working with folks at Lambda Labs to add support for their cloud. If other providers are interested, we'd be happy to chat.
(SkyPilot dev here)