Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

UC Berkeley launches SkyPilot to help navigate soaring cloud costs

datanami.com

Post reply on HN
Print view View original

51–60 of 114 posts

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#51

Post by laurencerowe »

nix23 wrote: post #49
laurencerowe wrote: post #47

Earlier quoted context omitted.

It really depends on the workload. GPU clusters are usually cheaper to run in house since Nvidia let you use regular GPUs for research which end up cheaper than cloud GPUs. And often Universities will charge less overhead for capital expenses on a grant which can artificially reduce the cost of running it yourself. The big downside of institutional HPC is it can be difficult to get stuff running on the ancient distri…

>since Nvidia let you use regular GPUs for research which end up cheaper than cloud GPUs. Funfact, that's illegal in Europe, producers have no right to tell you what to do with their products and your property.

While you own the hardware you don't own the software necessary to make use of it, just a license.

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#52

Post by quickthrower2 »

LilBytes wrote: post #17
manv1 wrote: post #15

Earlier quoted context omitted.

I've configured Cisco routers, F5s, LOMs, PDCs, NAS boxes, built servers from parts. And I'd never do it again. That shit has literally no value, and it's quite difficult to configure, manage, and maintain. And no matter how good your process, you (or your team member) inevitably will forget to go into the BIOS and turn off power saving...or forget to turn off (or on) proxy arp. Or you'll reboot a box and it won't co…

Same here. At massive scale, sure, it might make sense to run your own infrastructure and data centres. Some may even run their own private cloud and get the benefits we see from the public cloud today from their own IT depts. The vast majority of us however had nothing in common with the above. Like you we ran the hardware, and to the business, we were never more than a cost centre and anything pitched to the broade…

I am more the other way. Even the big clouds are too “serverful” for me, and for very small things a modern PaaS is much easier to deal with.

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#54

Post by covi »

laurencerowe wrote: post #50

Having worked on large compute intensive projects in both academia and industry I am somewhat sceptical about multi-cloud tools. Normally you do your compute where the data is since egress is so expensive. And it just doesn't seem worth developing against a cloud agnostic API and limiting myself to the lowest common denominator. If you have a large workload though they may be helpful as a negotiating tactic to play o…

One big reason people use multiple regions or clouds: higher resource availability.

Allocating scarce resources can be very hard on the cloud. These are things like high-end GPUs (both on-demand and spot; the latter being much harder to get) and also beefy CPU-based instances. We've seen 10s of hours of waiting times or longer.

The natural solution is to have the flexibility to allocate resources in multiple regions (and ultimately, clouds) to increase the total pool size.

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#55

Post by andbberger »

ww520 wrote: post #26
YetAnotherNick wrote: post #14

Earlier quoted context omitted.

Single ML job can’t be distributed very well. More often than not they hit network limit, even on single zone 1 GB/s network speed that we normally get. Most of the distributed workloads use something like NVLink.

That's true. Most ML algorithms have the iterate-until-converge pattern. How about some tasks like hyper parameter tuning or trying out different algorithms against the same data set? Those can be run in parallel. ML research should aim to produce more parallel algorithms.

state of the art models don't fit on a single gpu anymore, not for a while

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#56

Post by peterleiser »

why_only_15 wrote: post #22
ww520 wrote: post #7

I feel that the old SETI@home project can use a comeback. Most home machines have powerful GPU these days. The ML problems are embarrassingly parallel and can be distributed to the home machines. Just need to work out the economy for everyone involved.

Datacenter GPUs have >1000gbps network connections between nodes, which is necessary to actually utilize GPUs with current training techniques. It's possible that a furthering of techniques used in GPT-JT[1] might make it feasible to use home computers, but even GPT-JT requires at least a 1gbps connection. [1]: https://www.together.xyz/blog/releasing-v1-of-gpt-jt-powered...

Good point. One criterion for whether a SETI@home approach is reasonable for a class of problems is if the total processing time exceeds the network download and upload time. In SETI@home classic the work units didn't take too much time to download compared to how long it took to process a work unit.

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#57

Post by NavinF »

nix23 wrote: post #49
laurencerowe wrote: post #47

Earlier quoted context omitted.

It really depends on the workload. GPU clusters are usually cheaper to run in house since Nvidia let you use regular GPUs for research which end up cheaper than cloud GPUs. And often Universities will charge less overhead for capital expenses on a grant which can artificially reduce the cost of running it yourself. The big downside of institutional HPC is it can be difficult to get stuff running on the ancient distri…

>since Nvidia let you use regular GPUs for research which end up cheaper than cloud GPUs. Funfact, that's illegal in Europe, producers have no right to tell you what to do with their products and your property.

I swear, every thread has at least one European bragging about something ridiculous like "fun fact, murder is illegal in Europe" without spending 10 seconds to check if it's also illegal in the US.

Yes, we have property rights in the US. No, it's not obvious whether buying the hardware also gives you a license to use Nvidia's CUDA libraries without any limitations. Nobody has tested this in court.

Personally, I've got many consumer GPUs in my data center. If Nvidia doesn't like that, they can sue me. Username is real name.

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#59

Post by styren »

walrus01 wrote: post #4

At some point people will rediscover that "buying" whole sets of equipment through a lease financing company on a lease-to-own plan with $1 end payment, and colocating it the traditional way can often be significantly cheaper than paying endless "cloud" costs. naysayers will say, but the cloud allows you abstract away your salary costs of engineers! look at all the people you aren't hiring! I say: If your needs are s…

There's an alternative path where cloud companies actually start to compete on price. Right now you will pay gigantic premiums on your AWS infrastructure just by virtue of being in their ecosystem.

Personally, I see Kubernetes having huge potential to drive down costs. The ecosystem is still young but it's still possible to run such a wide range of services and tools on top of it, and it can easily be run on-prem or using managed providers.

I personally am part of a team building a much cheaper k8s service as I think costs have grown out of proportion and k8s offers the first real good shot at challenging AWS on cloud & costs.

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#60

Post by laurencerowe »

covi wrote: post #54
laurencerowe wrote: post #50

Having worked on large compute intensive projects in both academia and industry I am somewhat sceptical about multi-cloud tools. Normally you do your compute where the data is since egress is so expensive. And it just doesn't seem worth developing against a cloud agnostic API and limiting myself to the lowest common denominator. If you have a large workload though they may be helpful as a negotiating tactic to play o…

One big reason people use multiple regions or clouds: higher resource availability. Allocating scarce resources can be very hard on the cloud. These are things like high-end GPUs (both on-demand and spot; the latter being much harder to get) and also beefy CPU-based instances. We've seen 10s of hours of waiting times or longer. The natural solution is to have the flexibility to allocate resources in multiple regions…

I have seen very large jobs that needed to be split up across multiple regions but doing so required replicating large amounts of data. That comes at a cost too so it requires planning to work out which regions you want to use.

Once that is done it is fairly easy to set up spot clusters in each region to process the jobs in the queue.

Post reply on HN