Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

minimaxir.com

Post reply on HN
Print view View original

81–90 of 108 posts

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#81

Post by user5994461 »

boulos wrote: post #72
asah wrote: post #63

Earlier quoted context omitted.

Just signed up and ported my model + data: - it's indeed noticeably faster than the Google VMs. As usual, I compiled tensorflow for this GPU vs K80 (feature 6.1 vs 3.7). - ubuntu 16 minimal is indeed "minimal" ! but it worked... - GTX 1080 (7.92GB) has less GPU RAM than the K80 (11.17GiB) - this required me to reduce the model design slightly. For my model/data, Hetzner runs 1 training epoch in 1 hr vs 1.75 hr for Go…

This is presumably just the full board versus half nomenclature noted above. But yes, consumer GPUs are way more cost competitive than Tesla class parts. Being able to train bigger models is valuable to some folks, but not everyone, so I don't begrudge using the GTX line. Disclosure: I work on Google Cloud.

Unless I am mistaking, Tesla have ECC memory and consumers cards don't.

Should do some marketing on the disastrous effects it can have on the training.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#82

Post by beagle3 »

0xbear wrote: post #35
edwinksl wrote: post #33

Earlier quoted context omitted.

Would hyperthreading be helpful or harmful?

Hyper threading is not harmful per se. It lets your CPU make forward progress when it would otherwise be stalled waiting for something. My issue is that they call hyperthreads "vCPU" which makes it seem like you're getting a full core, while in reality you're getting 60% of a core at most.

Hyper threading often is harmful when you use it, because while it does let your CPU make forward progress, it does that at the expense of e.g. cache that is evicted.

Obviously depends on your workload, but on my highly parallel "standard" workloads, my experience is that you can get at most 15% more with hyperthreading on (e.g. 4 cores/8 threads) compared to off (4 cores/4 threads), whereas on the cache intensive loads, I get 20-30% LESS with hyperthreading on.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#83

Post by girvo »

lewisl9029 wrote: post #70
girvo wrote: post #31

Earlier quoted context omitted.

If you upvote it, you can find it again through your profile. I'm on mobile but I think you can favourite comments too via the time stamp link, also shows on your profile

I do this too, but does anyone have a good way of searching through your own upvoted comments/stories? A lot of times I couldn't find what a comment/story I know I saved because I've upvoted pages upon pages more stuff since.

What I do is for really important comments, the timestamp link -> "favourite" functionality gets used. Much smaller list :)

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#85

Post by XCSme »

shusson wrote: post #12
paulsutter wrote: post #2

Shoutout for Hetzner's 99 euro/month server with a GTX 1080, much better than the pseudo-K80s that Google Cloud provides for $520/month. The Google K80s are half or quarter the speed of a real K80, part of the reason they show so badly in the comparison. https://www.hetzner.com/dedicated-rootserver/ex51-ssd-gpu?co...

> Google K80s are half or quarter the speed of a real K80 do you have any evidence for this?

They clearly state this, the K80 contains two "GPUs", when you get one K80 instance you only get one of those two GPUs, so you get half of a K80.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#86

Post by ryanlol »

nevi-me wrote: post #67
icelancer wrote: post #47

Earlier quoted context omitted.

GTX 1080 GPU, i7-6700 Skylake, 64 GB DDR4 RAM, 2x500 GB 6 Gb/s SSDs for 99EUR/month with a one-time 99EUR setup fee. My lord. HPC resources are incredibly affordable. Hetnzer and some of the other dedicated server companies in Europe/Canada have some amazing deals (we've used OVH in the past with great success, and right now we use Paperspace for CPU intensive stuff we want to share expensive licensing on, like Visua…

Wow, to go off topic; I've been using Versaweb for the past 3-4 years, but became very unhappy after they forced a "server management" fee down our throats. We've been looking at setting up a small cluster of servers at work (budget of about $500), and I was still going to go with Versaweb. After seeing Hetzner, I'm going to reassess, and likely move everything there. I'm paying €150 for what it seems I could pay €10…

Why are you comparing Hetzner and Versaweb? They exist in completely different markets.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#87

Post by drieddust »

paulsutter wrote: post #2

Shoutout for Hetzner's 99 euro/month server with a GTX 1080, much better than the pseudo-K80s that Google Cloud provides for $520/month. The Google K80s are half or quarter the speed of a real K80, part of the reason they show so badly in the comparison. https://www.hetzner.com/dedicated-rootserver/ex51-ssd-gpu?co...

Though I never get to use Hetzner sop can't comment how good they are, I got into issues with them because of there convoluted process.

I was trying to calculate the total cost as their list price excludes VAT. It turns out they just booked the server for me and started sending invoice. Of course they allow to cancel within 14 days but I was handling a personal issue so didn't check my emails for almost a month. It turned messy.

If Hetzner support are listening, please improve the process and if possible take to credit card/payment details upfront so that person is aware that you are spinning the server for them.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#88

Post by nl »

user5994461 wrote: post #81
boulos wrote: post #72

Earlier quoted context omitted.

This is presumably just the full board versus half nomenclature noted above. But yes, consumer GPUs are way more cost competitive than Tesla class parts. Being able to train bigger models is valuable to some folks, but not everyone, so I don't begrudge using the GTX line. Disclosure: I work on Google Cloud.

Unless I am mistaking, Tesla have ECC memory and consumers cards don't. Should do some marketing on the disastrous effects it can have on the training.

Or.. just call it random dropout and market it as a feature.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#89

Post by tjoff »

mrb wrote: post #48
icelancer wrote: post #47

Earlier quoted context omitted.

GTX 1080 GPU, i7-6700 Skylake, 64 GB DDR4 RAM, 2x500 GB 6 Gb/s SSDs for 99EUR/month with a one-time 99EUR setup fee. My lord. HPC resources are incredibly affordable. Hetnzer and some of the other dedicated server companies in Europe/Canada have some amazing deals (we've used OVH in the past with great success, and right now we use Paperspace for CPU intensive stuff we want to share expensive licensing on, like Visua…

" incredibly affordable " It could go a lot lower. Hetzner's profit margin on renting a server like this for 99€/month is formidable.

How to you figure?

If I were to buy such as system it would be over €2000, that's not including cooling or a case for it either. Granted I live in Sweden so taxes are a bit on the high side.

Regardless, that will be at least 20 months before they make a dime (assuming they can rent it 100% of the time). And in that time it will collect rackspace along with electricity, bandwitdh (2 gbits and 50 TB per month) and a dedicated IP.

And after all that time that computer is not that hot anymore, but still draws just as much electricity regardless.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#90

Post by paulsutter »

boulos wrote: post #69
paulsutter wrote: post #2

Shoutout for Hetzner's 99 euro/month server with a GTX 1080, much better than the pseudo-K80s that Google Cloud provides for $520/month. The Google K80s are half or quarter the speed of a real K80, part of the reason they show so badly in the comparison. https://www.hetzner.com/dedicated-rootserver/ex51-ssd-gpu?co...

Just to reiterate barrus's (the Product Manager for K80s) point, K80s come with two dies per board, so we're giving you the granularity. We struggled with wording, but as both NVIDIA and AMD ping pong between GPUs with two dies per board as the best part versus one we didn't want to make the minimum granularity "a part sold by a vendor". So there's no conspiracy or half or quarter speed nonsense, just that it's proba…

You could survey your customers to see how widely this is understood. Likewise "cores"/hyperthreads.

Meanwhile the 10x price performance difference is the main point. Really eager to see the TPUs rolled out broadly, please do price them to take market share from NVIDIA

Post reply on HN