Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My low cost provider of GPUs has run out of capacity. Good alternatives?

hetzner.com

Post reply on HN
Print view View original

81–90 of 92 posts

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#81

Post by tech_man7 »

etaioinshrdlu wrote: post #65
tech_man7 wrote: post #60

Golem has GPUs for very cheap on its network so ask here https://www.reddit.com/r/GolemProject/ or https://twitter.com/golemproject?lang=en . Golem website is www.golem.network

I don't like that website, it looks almost like a blockchain ICO site. Low on details, high on hype. I don't see prices listed. Something seems wrong. This company does something pretty similar minus the blockchain and is much more straightforward: https://www.zerosix.ai/ , same with https://vast.ai/console/create/

It's not one of those ICO scams. Maybe you'd prefer their GitHub page in which the team has written over 100,000 lines of code. https://github.com/golemfactory/golem.

It's a marketplace so you name your own price and at the moment, it's used for CGI rendering, but next week there are more use cases being added, specifically WASM.

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#82

Post by pixelwhale »

coleca wrote: post #79
blodovnik wrote: post #37

Aws and Google spot instances are easily comparable price wise to hetzner but it really depends on what you are doing, which you haven't said. For example a g3 spot instance can be as low as 19 cents per hour $138/month. Google GPU instances I've run for 14 cents an hour. I think these aren't the lowest prices either.

AWS spot instances for gpu can have availability issues as well or sometimes be priced higher than on demand (still trying to understand that one). If you do choose to run spot at aws (gpu or otherwise) be sure to check out the excellent project at autospotting.org and donate if you use it. Makes it super easy to replace on demand nodes in an ASG with spot nodes and always make sure you’re getting a good price.

> AWS spot instances for gpu can have availability issues as well or sometimes be priced higher than on demand (still trying to understand that one).

You always pay the spot market price, not your bid. Your instance gets killed if somebody outbids you. A higher bid increases the probability that your instances don't get killed while at the same time letting you pay spot market price.

By bidding above on-demand price, you are speculating that for the majority of the time, nobody else will bid more than the on-demand price. If you're not the only one doing that, spot price can rise above on-demand price.

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#83

Post by minimaxir »

streetcat1 wrote: post #78
minimaxir wrote: post #56

Earlier quoted context omitted.

For one-time GPU processing tasks that only take a couple hours, AWS spot / GCP preemptible is far more cost efficient than Hetzner. However, if you need to run a GPU 24/7 and/or have massive bandwidth requirements, Hetzner is far more cost effective than AWS/GCP. It's a balance of needs.

Another option is to over provisioned spot instances, such that it would feel like a 24/7 instance, but cost like spot. However, you would probably need an automatic platform to achieve that.

An idea I've had is to use GCP preemptibles + script to automatically run a ML training job on instance start + Google Cloud Function + Cloud Scheduler to attempt to start the instance every minute if it gets preempted. The latter two are effectively free, so you'd get the cost benefits of preemptibles as long as the ML training job is resilient to random shutdowns.

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#84

Post by streetcat1 »

minimaxir wrote: post #83
streetcat1 wrote: post #78

Earlier quoted context omitted.

Another option is to over provisioned spot instances, such that it would feel like a 24/7 instance, but cost like spot. However, you would probably need an automatic platform to achieve that.

An idea I've had is to use GCP preemptibles + script to automatically run a ML training job on instance start + Google Cloud Function + Cloud Scheduler to attempt to start the instance every minute if it gets preempted. The latter two are effectively free, so you'd get the cost benefits of preemptibles as long as the ML training job is resilient to random shutdowns.

Yes, this is basically the idea. However, there are different solutions for training and inference. For training, I would recommend that you add automatic checkpoint, and even consider model migration. For inference (which I think is the original concern), over provisioning is the key (simply because the fact that it would take a long time to load the model. Also, you also want to diversify your node types, etc.

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#85

Post by matthew-wegner »

bitL wrote: post #63
matthew-wegner wrote: post #59

Elsewhere in the thread, it sounds like your needs a little more robust. But for anyone reading who might be interested: - Tesla K40 12GB cards are ~$130 on US eBay, and have been for awhile - These cards are passive, though, intending to be used in wind tunnel servers - However, you can put on 40mm screamer fans to cool them in a normal desktop. Something like https://www.thingiverse.com/thing:3032044 or just wire-t…

K80 are for ~$400, 24GB, 2xGPU, newer architecture, a bit better choice.

It shows up to the OS as two 12GB cards still. So it kind of depends on whether the higher cost is worth the extra slot (in my case it isn't). Total power is less than 2X on the card versus one K40, although I'm not sure offhand if that indicates less performance or just power savings from shared components.

As mentioned in a peer comment, K40 aren't the fastest cards either, but the 12GB is really nice for some use cases.

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#86

Post by bitL »

matthew-wegner wrote: post #85
bitL wrote: post #63

Earlier quoted context omitted.

K80 are for ~$400, 24GB, 2xGPU, newer architecture, a bit better choice.

It shows up to the OS as two 12GB cards still. So it kind of depends on whether the higher cost is worth the extra slot (in my case it isn't). Total power is less than 2X on the card versus one K40, although I'm not sure offhand if that indicates less performance or just power savings from shared components. As mentioned in a peer comment, K40 aren't the fastest cards either, but the 12GB is really nice for some use…

I have one K40 as well; you should be fine, even Haas F1 team built a CFD supercomputer full of them fairly recently ;-) Kepler is pretty good in FP64.

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#87

Post by ServerHunter »

chx wrote: post #40

https://www.serverhunter.com/?search=BF6-4AC-E30 says the cheapest provider with a GTX GPU is Ikoula with an 1070 at 101.66 USD and then Hetzner and then Hostkey. https://www.hostkey.com/gpu-servers It doesn't find https://dedispec.com/gpu.php these but I do :) I also read somewhere they even accept shipped in GPUs. I know they did with disks.

Sorry about that! While we had the Dedispec GPU servers listed, our spider system didn't correctly detect the presence of a GPU, so it didn't show up as such. It has now been resolved and they show up in your search results.

Thanks for using Server Hunter! :)

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#88

Post by etaioinshrdlu »

chx wrote: post #40

https://www.serverhunter.com/?search=BF6-4AC-E30 says the cheapest provider with a GTX GPU is Ikoula with an 1070 at 101.66 USD and then Hetzner and then Hostkey. https://www.hostkey.com/gpu-servers It doesn't find https://dedispec.com/gpu.php these but I do :) I also read somewhere they even accept shipped in GPUs. I know they did with disks.

Dedispec also appears to be out of stock! Oh no!

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#89

Post by etaioinshrdlu »

dragandj wrote: post #21
mciancia wrote: post #10

I wonder how are they dealing with nvidia EULAs about using geforce series cards in datacenters. Maybe that is the issue?

I've read somewhere (maybe in HN comments) that these kinds of clauses can not be enforced in EU. Hetzner operates out of Germany.

Yes, I wonder how sure HN is about this however!

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#90

Post by etaioinshrdlu »

ServerHunter wrote: post #87
chx wrote: post #40

https://www.serverhunter.com/?search=BF6-4AC-E30 says the cheapest provider with a GTX GPU is Ikoula with an 1070 at 101.66 USD and then Hetzner and then Hostkey. https://www.hostkey.com/gpu-servers It doesn't find https://dedispec.com/gpu.php these but I do :) I also read somewhere they even accept shipped in GPUs. I know they did with disks.

Sorry about that! While we had the Dedispec GPU servers listed, our spider system didn't correctly detect the presence of a GPU, so it didn't show up as such. It has now been resolved and they show up in your search results. Thanks for using Server Hunter! :)

Also missing some GPU servers from https://www.hostkey.com/gpu-servers ... although you got some of them.
Post reply on HN