Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My low cost provider of GPUs has run out of capacity. Good alternatives?

hetzner.com

Post reply on HN
Print view View original

71–80 of 92 posts

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#71

Post by Proven »

pbhjpbhj wrote: post #67
pantalaimon wrote: post #52

Earlier quoted context omitted.

Courts in Germany have ruled that you can not add additional clauses to a contract after the contract has been signed, e.g. the purchase has been made. So unless Hetzner is buying directly from nvidia, nvidia has no way to enforce such a clause. And when I buy a nvidia card at any retailer in Germany, I will not have to sign a contract that obligates me to not use it in a data center - heck I can just take the box of…

It's so good to hear when a legal system gets things right for consumers.

A good legal system is supposed to get things right, whether it's for consumer or producer.

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#72

Post by etaioinshrdlu »

somuchtyler wrote: post #68
etaioinshrdlu wrote: post #64

Earlier quoted context omitted.

The main use case is on demand neural net inference 24/7 availability. Kind of like hosting a website that must be on 24/7, this neural net must be on 24/7. It runs on the order of 50x faster on an nvidia GPU compared to a CPU. But it will sit idle most of the time. Price per hour per GB of GPU ram is the most important metric when choosing a server.

I really like nocix servers, they are out of Kansas City. They got their "i7-6700K 32GB + 2x 480GB SSD + GTX 1080" for $105/mo https://www.nocix.net/cart/?id=338

This sounds great but they are also out of stock!

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#73

Post by workingpatrick »

etaioinshrdlu wrote: post #65
tech_man7 wrote: post #60

Golem has GPUs for very cheap on its network so ask here https://www.reddit.com/r/GolemProject/ or https://twitter.com/golemproject?lang=en . Golem website is www.golem.network

I don't like that website, it looks almost like a blockchain ICO site. Low on details, high on hype. I don't see prices listed. Something seems wrong. This company does something pretty similar minus the blockchain and is much more straightforward: https://www.zerosix.ai/ , same with https://vast.ai/console/create/

It's exactly a blockchain ICO site :D https://golem.network/crowdfunding/

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#74

Post by Johnny555 »

pbhjpbhj wrote: post #66
bitL wrote: post #63

Earlier quoted context omitted.

K80 are for ~$400, 24GB, 2xGPU, newer architecture, a bit better choice.

You consider $400 'low cost'?

The article link mentions an NVIDEO card, and an NVIDIA GeForce GTX 1080 8GB sells for ~ $750 new. I don't know how this $400 card compares for this use case, but it is relatively cheap.

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#75

Post by etaioinshrdlu »

rpedela wrote: post #70
etaioinshrdlu wrote: post #64

Earlier quoted context omitted.

The main use case is on demand neural net inference 24/7 availability. Kind of like hosting a website that must be on 24/7, this neural net must be on 24/7. It runs on the order of 50x faster on an nvidia GPU compared to a CPU. But it will sit idle most of the time. Price per hour per GB of GPU ram is the most important metric when choosing a server.

Would AWS Elastic Inference work for your use case? https://aws.amazon.com/machine-learning/elastic-inference/

No, I need NVIDIA CUDA support. Also, the most important metric for me is cost per month per GB of GPU RAM, and AWS elastic inference is pretty bad in that metric.

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#76

Post by bitL »

Johnny555 wrote: post #74
pbhjpbhj wrote: post #66

Earlier quoted context omitted.

You consider $400 'low cost'?

The article link mentions an NVIDEO card, and an NVIDIA GeForce GTX 1080 8GB sells for ~ $750 new. I don't know how this $400 card compares for this use case, but it is relatively cheap.

K80 is about as fast as 1080 in FP32 (8.2 vs 8.8TFlops) but vastly faster in FP64 (2.7 vs 0.28TFlops; FP64 supercomputers are still being built with it). It has also 3x the memory, so fitting BERT_large NLP model might be possible. 1080 is pretty much outdated at this point as not many state-of-art models can fit inside 8GB. Disadvantage of K80 are older CUDA kernels versions so customized kernels for new CUDA versions might not work, but most models don't touch CUDA directly anyway.

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#77

Post by monksy »

cryptofits wrote: post #22

Hetzner support is pretty bad took them like 6 days to answer my ticket

My firend's experience has been the same. We've had issues where our wordpress instance was exploited and we got kicked off the network.

Also, we've had issues where there was a upnp connection open and we got kicked off the network for that as well.

In those cases it took days for them to get back via email, and then they struggled to open up a channel which we could resolve the issue and get it compliant.

Also, we've had situations where there was maintenance but no update that it was going on.

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#78

Post by streetcat1 »

minimaxir wrote: post #56
blodovnik wrote: post #37

Aws and Google spot instances are easily comparable price wise to hetzner but it really depends on what you are doing, which you haven't said. For example a g3 spot instance can be as low as 19 cents per hour $138/month. Google GPU instances I've run for 14 cents an hour. I think these aren't the lowest prices either.

For one-time GPU processing tasks that only take a couple hours, AWS spot / GCP preemptible is far more cost efficient than Hetzner. However, if you need to run a GPU 24/7 and/or have massive bandwidth requirements, Hetzner is far more cost effective than AWS/GCP. It's a balance of needs.

Another option is to over provisioned spot instances, such that it would feel like a 24/7 instance, but cost like spot. However, you would probably need an automatic platform to achieve that.

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#79

Post by coleca »

blodovnik wrote: post #37

Aws and Google spot instances are easily comparable price wise to hetzner but it really depends on what you are doing, which you haven't said. For example a g3 spot instance can be as low as 19 cents per hour $138/month. Google GPU instances I've run for 14 cents an hour. I think these aren't the lowest prices either.

AWS spot instances for gpu can have availability issues as well or sometimes be priced higher than on demand (still trying to understand that one).

If you do choose to run spot at aws (gpu or otherwise) be sure to check out the excellent project at autospotting.org and donate if you use it. Makes it super easy to replace on demand nodes in an ASG with spot nodes and always make sure you’re getting a good price.

Re: My low cost provider of GPUs has run out of capacity. Good alternatives?

#80

Post by pixelwhale »

bitL wrote: post #76
Johnny555 wrote: post #74

Earlier quoted context omitted.

The article link mentions an NVIDEO card, and an NVIDIA GeForce GTX 1080 8GB sells for ~ $750 new. I don't know how this $400 card compares for this use case, but it is relatively cheap.

K80 is about as fast as 1080 in FP32 (8.2 vs 8.8TFlops) but vastly faster in FP64 (2.7 vs 0.28TFlops; FP64 supercomputers are still being built with it). It has also 3x the memory, so fitting BERT_large NLP model might be possible. 1080 is pretty much outdated at this point as not many state-of-art models can fit inside 8GB. Disadvantage of K80 are older CUDA kernels versions so customized kernels for new CUDA versio…

Being a dual-GPU model, it really can only fit half the total memory in the majority of cases, because sharing memory across GPUs is far slower and more difficult to implement.
Post reply on HN