Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

minimaxir.com

Post reply on HN
Print view View original

71–80 of 108 posts

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#71

Post by 0xbear »

dkobran wrote: post #61

One of the interesting variables in calculating ML training costs is developer time. The cost of a Data Scientist (or similar role) on an hourly basis will far outweigh the most expensive compute resource by several orders of magnitude. When you factor in time, the GPU immediately becomes more attractive. Other industries with heavy/time consuming computational workloads like CGI rendering have understood this for de…

For training, that's likely to be true. For large scale inference it's not possible to beat CPUs right now if cost is a factor. You might be able to beat them once you can buy TPU access in cloud, depending on how steep a premium Google attaches to it.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#72

Post by boulos »

asah wrote: post #63
paulsutter wrote: post #2

Shoutout for Hetzner's 99 euro/month server with a GTX 1080, much better than the pseudo-K80s that Google Cloud provides for $520/month. The Google K80s are half or quarter the speed of a real K80, part of the reason they show so badly in the comparison. https://www.hetzner.com/dedicated-rootserver/ex51-ssd-gpu?co...

Just signed up and ported my model + data: - it's indeed noticeably faster than the Google VMs. As usual, I compiled tensorflow for this GPU vs K80 (feature 6.1 vs 3.7). - ubuntu 16 minimal is indeed "minimal" ! but it worked... - GTX 1080 (7.92GB) has less GPU RAM than the K80 (11.17GiB) - this required me to reduce the model design slightly. For my model/data, Hetzner runs 1 training epoch in 1 hr vs 1.75 hr for Go…

This is presumably just the full board versus half nomenclature noted above. But yes, consumer GPUs are way more cost competitive than Tesla class parts. Being able to train bigger models is valuable to some folks, but not everyone, so I don't begrudge using the GTX line.

Disclosure: I work on Google Cloud.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#73

Post by tempay »

shusson wrote: post #22
barrus wrote: post #20

Earlier quoted context omitted.

Each Google K80s is one GPU or 1/2 of a K80 board, so technically you are correct that a Google K80 GPU is half of a K80 board. However, they are offered in passthrough mode and achieve full performance. If you want a whole K80 board, attach 2 K80 GPUs to a single VM. You can have 1, 2, 4 or 8 K80 GPUs attached to each VM in GCP. (I'm one of the GPU product managers at Google Cloud).

thanks for the info. On a side note, do you know if GCP will ever support preemptible GPU instances?

According to this[1] comment it's on their mind but doesn't seem like a priority.

[1] https://news.ycombinator.com/item?id=14728476

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#74

Post by likelynew »

shusson wrote: post #6

While the authors article is relevant if you are stuck on GCP, on AWS you will not have the same conclusion. This is because AWS has GPU spot instances (P2) which can be found for ~80% cheaper depending your region [1]. Hopefully one day soon GCP will support preemptible GPU instances. [1] https://aws.amazon.com/ec2/spot/pricing/

When I started it was even cheaper(~10% of reserved cost), but even now, it's pretty cheap.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#75

Post by likelynew »

cobookman wrote: post #14

I've been amazed that more people don't make use of googles preemtibles. Not only are they great for background batch compute. You can also use them for cutting your stateless webserver compute costs down. I've seen some people use k8s with a cluster of preemtibles and non preemtibles.

preemtibles are not available for GPU.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#76

Post by alexcnwy »

0xbear wrote: post #13

FYI, y'all: cloud "cores" are actually hyperthreads. Cloud GPUs are single dies on multi-die card. If you use GPUs 24x7, just buy a few 1080 Ti cards and forego the cloud entirely. If you must use TF in cloud with CPU, compile it yourself with AVX2 and FMA support. Stock TF is compiled for the lowest common denominator.

Hadn't heard about compiling yourself improving performance for cloud CPU usage - thanks!

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#77

Post by alexcnwy »

visarga wrote: post #58

For research and experimentation what you need is your own DL box. It will pay for itself in a few months. You will feel better having your own reliable hardware that you don't share or have to pay by the minute, and that will impact the kind of ideas you are going to try. Then you scale up to the cloud to do hyperparameter search.

Do you have any advice on getting your own box set up in a data center - constantly traveling...

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#79

Post by detaro »

shmageggy wrote: post #28
paulsutter wrote: post #2

Shoutout for Hetzner's 99 euro/month server with a GTX 1080, much better than the pseudo-K80s that Google Cloud provides for $520/month. The Google K80s are half or quarter the speed of a real K80, part of the reason they show so badly in the comparison. https://www.hetzner.com/dedicated-rootserver/ex51-ssd-gpu?co...

Wish HN had a 'save' feature so I can remember this comment when I need a GPU box.

click on the timestamp, click "favorite"

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#80

Post by detaro »

nickpsecurity wrote: post #56
shmageggy wrote: post #28

Earlier quoted context omitted.

Wish HN had a 'save' feature so I can remember this comment when I need a GPU box.

You can hit the favorite link at the top of the thread so it's in your favorites.

individual comments have "favorite" as well -> just click on the timestamp to get to the single-comment view.
Post reply on HN