Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

UC Berkeley launches SkyPilot to help navigate soaring cloud costs

datanami.com

Post reply on HN
Print view View original

101–110 of 114 posts

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#101

Post by mdaniel »

mlboss wrote: post #89
donnygreenberg wrote: post #39

I've been using SkyPilot for 3 months within an ML platform project and it is exactly as awesome as it sounds. The overall experience for launching and managing compute is thoughtful and ergonomic in a way that we've been conditioned not to expect from cloud compute (let alone GPU compute). It was a real shock to start using it, and now every time I need to jump back into the weeds of a cloud provider's APIs I'm remi…

Does it support spot instances ?

they'd be crazy not to, unless the ML pipeline didn't support retries: https://skypilot.readthedocs.io/en/latest/examples/spot-jobs...

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#102

Post by yummypaint »

bombcar wrote: post #84
master_crab wrote: post #78

Earlier quoted context omitted.

Most people don’t bother refactoring their application and leave whatever IIS garbage they have running on windows 20xx in a cloud instance (to say nothing about the windows licensing cost). Also universities don’t have the same cost models as companies. Their labs have less money - except those institutions with large endowments - but they all have access to near limitless “slave” labor in the form of grad students…

When I was involved with univs the labs were bursting at the seams with hardware, because companies abounded who loved nothing more than dumping product on us at cheap or free, especially if we'd get students using it.

I'm at a university lab and would love some free computing and storage hardware. Any tips?

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#104

Post by jsd_dmatrix »

donnygreenberg wrote: post #39

I've been using SkyPilot for 3 months within an ML platform project and it is exactly as awesome as it sounds. The overall experience for launching and managing compute is thoughtful and ergonomic in a way that we've been conditioned not to expect from cloud compute (let alone GPU compute). It was a real shock to start using it, and now every time I need to jump back into the weeds of a cloud provider's APIs I'm remi…

Underneath, Skypilot uses a Ray quite a bit. You can read is paper here: https://sigops.org/s/conferences/hotos/2021/papers/hotos21-s...

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#105

Post by KronisLV »

bluedino wrote: post #97
KronisLV wrote: post #43

Aren't AWS, Azure and GCP all comparatively expensive? They are good for certain workloads and it's good that this project exists, but if you wanted cheap cloud resources, you'd probably need to look at one of the smaller cloud vendors: Hetzner, Scaleway, DigitalOcean, Vultr, Contabo and others. Of course, if you need GPUs, things can get limiting.

Does DO or Vultr even offer the HPC instances AWS does? 96 CPU, 384GB, etc? Let's not even mention storage or networking.

Such configurations sadly aren't relevant for me, because neither are any of my workloads that demanding, nor could I ever afford to rent such servers anyways. At that point, one might as well make the argument for forking over money towards the more "enterprise" platforms.

However, out of curiosity, I actually checked all of the mentioned platforms, to figure out where they cap out.

Cloud services:

  Contabo - CLOUD VDS XXL has 12 CPU and 96 GB of RAM, 149 euros/month
  Hetzner - CCX62 has 48 CPU and 192 GB of RAM, 532 euros/month
  DigitalOcean - caps out at around 32 CPU and 256 GB of RAM, 2096 euros/month
  Scaleway - ENT1-2XL has 96 CPU and 384 GB of RAM, 2576 euros/month
  Vultr - caps out at around 96 CPU and 255 GB of RAM, 3840 euros/month
Dedicated servers:

  Contabo - AMD EPYC 32 Cores has 32 CPU and 256 GB of RAM, 249 euros/month
  Hetzner - AX161 has 32 CPU and 1024 GB of RAM, 833 euros/month
  Scaleway - EM-T210E-NVME has 128 CPU and 2048 GB of RAM, 2625 euros/month
  Vultr - NVIDIA A100 has 96 CPU and 960 GB of RAM, 14000 euros/month
So to answer that question, yes, most of the platforms out there have more performant offerings, while the prices and details vary. It's great that these alternatives exist, even if the needs of most people will be more modest.

For example, my hybrid container cluster with 6 nodes has 13 CPU cores in and a total of 54 GB of RAM and even that's with room for growth, hosting basically everything that I need for about 30 euros a month. Most people out there will probably care about finding ways to cheaply run their WordPress site or something, as opposed to scaling way up.

That's kind of the beauty of the current market - there's something for everyone out there!

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#106

Post by bombcar »

yummypaint wrote: post #102
bombcar wrote: post #84

Earlier quoted context omitted.

When I was involved with univs the labs were bursting at the seams with hardware, because companies abounded who loved nothing more than dumping product on us at cheap or free, especially if we'd get students using it.

I'm at a university lab and would love some free computing and storage hardware. Any tips?

Write letters to companies you're interested in. You should have a relatively standard form letter explaining who you are, what the university is, how great it is, etc, etc. We got a moderate hit rate on those (higher if you consider discounts).

Look through donor and alumni records for big name companies.

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#107

Post by _a_a_a_ »

yummypaint wrote: post #102
bombcar wrote: post #84

Earlier quoted context omitted.

When I was involved with univs the labs were bursting at the seams with hardware, because companies abounded who loved nothing more than dumping product on us at cheap or free, especially if we'd get students using it.

I'm at a university lab and would love some free computing and storage hardware. Any tips?

Theft.

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#108

Post by bauruine »

manv1 wrote: post #15
walrus01 wrote: post #4

At some point people will rediscover that "buying" whole sets of equipment through a lease financing company on a lease-to-own plan with $1 end payment, and colocating it the traditional way can often be significantly cheaper than paying endless "cloud" costs. naysayers will say, but the cloud allows you abstract away your salary costs of engineers! look at all the people you aren't hiring! I say: If your needs are s…

I've configured Cisco routers, F5s, LOMs, PDCs, NAS boxes, built servers from parts. And I'd never do it again. That shit has literally no value, and it's quite difficult to configure, manage, and maintain. And no matter how good your process, you (or your team member) inevitably will forget to go into the BIOS and turn off power saving...or forget to turn off (or on) proxy arp. Or you'll reboot a box and it won't co…

I do 2.9 PB a month for less than 1000$. And this is for rented servers so even includes the hardware. That would be more than 30k$ at your rate. Not going to argue about the rest but the cloud is charging horrendous rates for egress even with negotiated discounts.

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#109

Post by noogle »

manv1 wrote: post #15
walrus01 wrote: post #4

At some point people will rediscover that "buying" whole sets of equipment through a lease financing company on a lease-to-own plan with $1 end payment, and colocating it the traditional way can often be significantly cheaper than paying endless "cloud" costs. naysayers will say, but the cloud allows you abstract away your salary costs of engineers! look at all the people you aren't hiring! I say: If your needs are s…

I've configured Cisco routers, F5s, LOMs, PDCs, NAS boxes, built servers from parts. And I'd never do it again. That shit has literally no value, and it's quite difficult to configure, manage, and maintain. And no matter how good your process, you (or your team member) inevitably will forget to go into the BIOS and turn off power saving...or forget to turn off (or on) proxy arp. Or you'll reboot a box and it won't co…

False dichotomy. The alternative to AWS is not building your own data center. It's renting bare-metal servers from Hetzner/DigitalOcean.

Zero hardware headaches with x5-10 lower cost.

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#110

Post by ritwikgupta »

m2fkxy wrote: post #85
ritwikgupta wrote: post #9

We’re using SkyPilot and Skyplane at Berkeley and the Department of Defense to scale our damage assessment from satellite imagery machine learning models [0] to run for all of Ukraine, all the time! We’re able to detect instances of war crimes rapidly and use them to help plan the reconstruction of Ukraine. [0] https://xview2.org/

do you use SAR?

We do not currently use SAR! High-resolution SAR is currently very hard to find. Vendors like Capella Space are pretty pricey for HADR operations, though they do provide some data for deeply discounted/free prices for major disasters.

Beyond cost, SAR is just a deeply different form of imaging. The underlying physics that define a SAR image are so different from EO that many computer vision methods, in my opinion, will need to be re-invented to accommodate it. In fact, the next good bit of my Ph.D. is dedicated to accomplishing just that.

Urban scenes look very chaotic in SAR due to multi-path and layover effects. Defining damage in that environment is more aggregate than individual buildings. We're trying to figure out a good way to define what damage even means on SAR. In fact, one of the biggest accomplishments of our xView2 work was being the first comprehensive effort to define what damage means, for any natural disaster, from EO satellite imagery!

Post reply on HN