Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

minimaxir.com

Post reply on HN
Print view View original

11–20 of 108 posts

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#11

Post by cvhvcc »

automatapr wrote: post #5

Neat article. I think it's worth pointing out that this guy is an active commenter in the Hackathon Hackers facebook group, if you want to see more of his content. He can be pretty pretentious sometimes, but good content nonetheless.

I've seen him link some funny/ridiculous conversations that take place in that group (eg- can people who develop wix websites be considered web developers?), but unfortunately I can't more content since I'm not on FB. If there's an archive of all the funny conversations, let me know.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#12

Post by shusson »

paulsutter wrote: post #2

Shoutout for Hetzner's 99 euro/month server with a GTX 1080, much better than the pseudo-K80s that Google Cloud provides for $520/month. The Google K80s are half or quarter the speed of a real K80, part of the reason they show so badly in the comparison. https://www.hetzner.com/dedicated-rootserver/ex51-ssd-gpu?co...

> Google K80s are half or quarter the speed of a real K80

do you have any evidence for this?

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#13

Post by 0xbear »

FYI, y'all: cloud "cores" are actually hyperthreads. Cloud GPUs are single dies on multi-die card. If you use GPUs 24x7, just buy a few 1080 Ti cards and forego the cloud entirely. If you must use TF in cloud with CPU, compile it yourself with AVX2 and FMA support. Stock TF is compiled for the lowest common denominator.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#14

Post by cobookman »

I've been amazed that more people don't make use of googles preemtibles. Not only are they great for background batch compute. You can also use them for cutting your stateless webserver compute costs down. I've seen some people use k8s with a cluster of preemtibles and non preemtibles.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#15

Post by yahyaheee »

brianwawok wrote: post #8
yahyaheee wrote: post #4

No spot instances?

This entire article was about spot instances. On GCE they are called preemptible.

Didn't catch this thanks, spot instances have a bit better rate, but as he mentioned they are billed by hour :)

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#16

Post by AndrewKemendo »

Excellent write-up, kudos on going through all of that Max. Too bad Google will deprecate the preemptable instances as a result :P.

There is a notable CPU-specific TensorFlow behavior; if you install from pip (as the official instructions and tutorials recommend) and begin training a model in TensorFlow, you’ll see these warnings in the console:

FWIW I get the console warnings with the Tensorflow-GPU installation from pip, and I verified that it was actually using the GPU.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#17

Post by brianwawok »

yahyaheee wrote: post #15
brianwawok wrote: post #8

Earlier quoted context omitted.

This entire article was about spot instances. On GCE they are called preemptible.

Didn't catch this thanks, spot instances have a bit better rate, but as he mentioned they are billed by hour :)

Yes premptiable are a much better deal with minute billing.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#18

Post by ohstopitu »

cobookman wrote: post #14

I've been amazed that more people don't make use of googles preemtibles. Not only are they great for background batch compute. You can also use them for cutting your stateless webserver compute costs down. I've seen some people use k8s with a cluster of preemtibles and non preemtibles.

something I've always been curious about (and if a Google Cloud Engineer could clear up - that would be great), is why we should not (as in, why does everyone not) use preemptible nodes (apart from maybe the 3 / 5 master nodes).

My question specifically being: if I configure a k8s cluster to have all my slaves as preemptible nodes...would GCP automatically add new nodes as my old nodes are deleted (from what I understand preemptible nodes are assigned to you for a max of 24 hrs)?

Considering the pricing of preemptible nodes + the discounts that GCP assigns to you for sustained use, it makes cloud insanely cheap for an early stage startup.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#19

Post by thesandlord »

ohstopitu wrote: post #18
cobookman wrote: post #14

I've been amazed that more people don't make use of googles preemtibles. Not only are they great for background batch compute. You can also use them for cutting your stateless webserver compute costs down. I've seen some people use k8s with a cluster of preemtibles and non preemtibles.

something I've always been curious about (and if a Google Cloud Engineer could clear up - that would be great), is why we should not (as in, why does everyone not) use preemptible nodes (apart from maybe the 3 / 5 master nodes). My question specifically being: if I configure a k8s cluster to have all my slaves as preemptible nodes...would GCP automatically add new nodes as my old nodes are deleted (from what I unders…

Google Cloud Developer Advocate here.

Go for it as long as you understand the downside. It's possible that all instances get preempted at once (especially at the 24hr mark), that there isn't capacity to spin up new preemptible nodes in the selected zone once the old instance is deleted, etc. New VMs also take time to boot and join the cluster.

If you are just doing dev/test stuff, I'd recommend using a namespace in your production cluster or spinning up and down test clusters on demand (which can be preemptible).

If you have long running tasks (like a database) or are serving production traffic, using 100% preemptible nodes is not a good idea.

Preemptible can be great for burst traffic and batch jobs, or you can do a mix of preemptible and standard to get the right mix of stability and cost.

Re: Benchmarking TensorFlow on Cloud CPUs: Cheaper Deep Learning Than Cloud GPUs

#20

Post by barrus »

shusson wrote: post #12
paulsutter wrote: post #2

Shoutout for Hetzner's 99 euro/month server with a GTX 1080, much better than the pseudo-K80s that Google Cloud provides for $520/month. The Google K80s are half or quarter the speed of a real K80, part of the reason they show so badly in the comparison. https://www.hetzner.com/dedicated-rootserver/ex51-ssd-gpu?co...

> Google K80s are half or quarter the speed of a real K80 do you have any evidence for this?

Each Google K80s is one GPU or 1/2 of a K80 board, so technically you are correct that a Google K80 GPU is half of a K80 board. However, they are offered in passthrough mode and achieve full performance. If you want a whole K80 board, attach 2 K80 GPUs to a single VM. You can have 1, 2, 4 or 8 K80 GPUs attached to each VM in GCP. (I'm one of the GPU product managers at Google Cloud).
Post reply on HN