Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TensorFlow Benchmarks

github.com

Post reply on HN
Print view View original

21–30 of 57 posts

Re: TensorFlow Benchmarks

#21

Post by sanxiyn »

aljones wrote: post #16
mark_l_watson wrote: post #10

From Google's perspective it is probably more about how TensorFlow scales out horizontally. If a researcher fires off a Borg run (or whatever they use now) and the job takes a few thousand CPUs, no problem, at least for research. They must have better optimization a for running in production, such as in place operations.

So how horizontally scalable is it?

As far as I know, TensorFlow's scaling behavior has not been published.

For its predecessor DistBelief, here are some quotes from NIPS 2012 paper:

"The moderately sized speech model runs fastest on 8 machines, computing 2.2x faster than using a single machine. (Models were configured to use no more than 20 cores per machine.) Partitioning the model on more than 8 machines actually slows training, as network overhead starts to dominate in the fully-connected network structure and there is less work for each machine to perform with more partitions." ("The moderately sized" model here has 42 million model parameters. Check the paper for details.)

"In contrast, the much larger, locally-connected image models can benefit from using many more machines per model replica. The largest model, with 1.7 billion parameters benefits the most, giving a speedup of more than 12x using 81 machines. For these large models using more machines continues to increase speed, but with diminishing returns."

http://research.google.com/pubs/pub40565.html

Re: TensorFlow Benchmarks

#22

Post by discardorama »

argonaut wrote: post #20
donthateyo wrote: post #13

Until now, I've seen two responses to Google's TensorFlow from Facebook employees. Yann Le Cunn seemed to really challenge Jeff Dean about TensorFlow's scalability [1] and this benchmark puts TensorFlow down there in all the measures it tested for. I can't ignore the possibility that this criticism of TensorFlow from Facebook employees (while factually correct and constructive) might be driven by some competition and…

I think I'll give the benefit of the doubt to, you know, the pioneer of deep learning, inventor of convolutional neural networks, and (co)-inventor of the backpropagation algorithm.

Just because you invent an algorithm, it doesn't mean you know how to implement it in the most efficient way possible.

I think the key to TensorFlow is not how fast it runs on 1 machine; but how fast it runs on 10,000.

Consider map-reduce (Hadoop). Sure, you can sort 1GB data on a single machine 10x faster (using /usr/bin/sort) than using Hadoop on that machine; but make the data 1TB and add 1000 machines, now lets see how fast you can sort with /usr/bin/sort!

Re: TensorFlow Benchmarks

#23

Post by argonaut »

discardorama wrote: post #22
argonaut wrote: post #20

Earlier quoted context omitted.

I think I'll give the benefit of the doubt to, you know, the pioneer of deep learning, inventor of convolutional neural networks, and (co)-inventor of the backpropagation algorithm.

Just because you invent an algorithm, it doesn't mean you know how to implement it in the most efficient way possible. I think the key to TensorFlow is not how fast it runs on 1 machine; but how fast it runs on 10,000. Consider map-reduce (Hadoop). Sure, you can sort 1GB data on a single machine 10x faster (using /usr/bin/sort) than using Hadoop on that machine; but make the data 1TB and add 1000 machines, now lets s…

I'm just pointing out how ridiculous it is to cast aspersions on the judgment of one of the world's leading experts on deep learning.

It's also getting out of hand because everyone has already decided TensorFlow must be amazing, so everyone is extolling the virtues of what right now we only know to be speculation and PR.

Re: TensorFlow Benchmarks

#24

Post by dave_sullivan »

mark_l_watson wrote: post #10

From Google's perspective it is probably more about how TensorFlow scales out horizontally. If a researcher fires off a Borg run (or whatever they use now) and the job takes a few thousand CPUs, no problem, at least for research. They must have better optimization a for running in production, such as in place operations.

I'm going to say something unpopular, but horizontally-scaled deep learning is overkill for most applications.

Can anyone here present a use case where they have personally needed horizontal scaling because a Titan X couldn't fit what they were trying to do? It's the thing I hear talked about the most and used the least.

The biggest misunderstanding I've heard is "I have petabytes of data so I need multi-GPU", but NNs are trained on batches of data, so it doesn't matter. It's the model that must fit, but 12gb can fit a big model.

The whole "who can build the biggest net" is like building the fastest car: it's for show, the slower ones are way more cost effective and get you there just as well.

Keras is my present deep learning library recommendation--it's like torch 7 but in python and theano. Multi-GPU is roadmap because there's a lot more going on in deep learning that acts as a bigger differentiator than horizontal scaling.

For the record, when I say scaling, I'm not talking about having multiple machines serve multiple requests (that's a good reason to purchase more machines), I'm talking about spreading your gradient computations and/or model weights across machines or GPUs.

Re: TensorFlow Benchmarks

#25

Post by freerobby »

dave_sullivan wrote: post #24
mark_l_watson wrote: post #10

From Google's perspective it is probably more about how TensorFlow scales out horizontally. If a researcher fires off a Borg run (or whatever they use now) and the job takes a few thousand CPUs, no problem, at least for research. They must have better optimization a for running in production, such as in place operations.

I'm going to say something unpopular, but horizontally-scaled deep learning is overkill for most applications. Can anyone here present a use case where they have personally needed horizontal scaling because a Titan X couldn't fit what they were trying to do? It's the thing I hear talked about the most and used the least. The biggest misunderstanding I've heard is "I have petabytes of data so I need multi-GPU", but NN…

Would image training for video recognition be a good example of this?

Re: TensorFlow Benchmarks

#26

Post by dave_sullivan »

freerobby wrote: post #25
dave_sullivan wrote: post #24

Earlier quoted context omitted.

I'm going to say something unpopular, but horizontally-scaled deep learning is overkill for most applications. Can anyone here present a use case where they have personally needed horizontal scaling because a Titan X couldn't fit what they were trying to do? It's the thing I hear talked about the most and used the least. The biggest misunderstanding I've heard is "I have petabytes of data so I need multi-GPU", but NN…

Would image training for video recognition be a good example of this?

It's a good example of something people think they need multi-GPU for, but no, I don't think it's needed. For the same reason as "I've got petabytes of data".

Re: TensorFlow Benchmarks

#27

Post by hueving »

argonaut wrote: post #23
discardorama wrote: post #22

Earlier quoted context omitted.

Just because you invent an algorithm, it doesn't mean you know how to implement it in the most efficient way possible. I think the key to TensorFlow is not how fast it runs on 1 machine; but how fast it runs on 10,000. Consider map-reduce (Hadoop). Sure, you can sort 1GB data on a single machine 10x faster (using /usr/bin/sort) than using Hadoop on that machine; but make the data 1TB and add 1000 machines, now lets s…

I'm just pointing out how ridiculous it is to cast aspersions on the judgment of one of the world's leading experts on deep learning. It's also getting out of hand because everyone has already decided TensorFlow must be amazing, so everyone is extolling the virtues of what right now we only know to be speculation and PR .

>I'm just pointing out how ridiculous it is to cast aspersions on the judgment of one of the world's leading experts on deep learning.

A position of authority doesn't mean the person is immune to jealousy. Someone in a high status position is going to be much more likely to have a knee-jerk defensive reaction to news that threatens their image.

Re: TensorFlow Benchmarks

#28

Post by kastnerkyle »

dave_sullivan wrote: post #24
mark_l_watson wrote: post #10

From Google's perspective it is probably more about how TensorFlow scales out horizontally. If a researcher fires off a Borg run (or whatever they use now) and the job takes a few thousand CPUs, no problem, at least for research. They must have better optimization a for running in production, such as in place operations.

I'm going to say something unpopular, but horizontally-scaled deep learning is overkill for most applications. Can anyone here present a use case where they have personally needed horizontal scaling because a Titan X couldn't fit what they were trying to do? It's the thing I hear talked about the most and used the least. The biggest misunderstanding I've heard is "I have petabytes of data so I need multi-GPU", but NN…

For big LSTMs and long-ish sequences, the intermediate gradients can take up a huge amount of memory - often more than the model parameters themselves. In my experience it is mostly big LSTMs that need the 12GB+ GPUs. You can reduce the batch size to help this a bit, or train using trucated BPTT but RNN training is already a slow, sequential business.

Of course, there are no clear wins (generally, losses) in computation by scaling RNNs horizontally on GPUs - but sometimes you really do need more than 12GB.

Re: TensorFlow Benchmarks

#29

Post by kastnerkyle »

hueving wrote: post #27
argonaut wrote: post #23

Earlier quoted context omitted.

I'm just pointing out how ridiculous it is to cast aspersions on the judgment of one of the world's leading experts on deep learning. It's also getting out of hand because everyone has already decided TensorFlow must be amazing, so everyone is extolling the virtues of what right now we only know to be speculation and PR .

>I'm just pointing out how ridiculous it is to cast aspersions on the judgment of one of the world's leading experts on deep learning. A position of authority doesn't mean the person is immune to jealousy. Someone in a high status position is going to be much more likely to have a knee-jerk defensive reaction to news that threatens their image.

Yann LeCun has been working on hardware implementations of convnets almost since the beginning. LeNet 5 (the check reader of ATT, circa early 90s) needed a dedicated and specialized hardware implementation IIRC. It could be kneejerk, but he definitely has the background to make a fair assessment - and this benchmark seems to back his claims.

Re: TensorFlow Benchmarks

#30

Post by kastnerkyle »

donthateyo wrote: post #13

Until now, I've seen two responses to Google's TensorFlow from Facebook employees. Yann Le Cunn seemed to really challenge Jeff Dean about TensorFlow's scalability [1] and this benchmark puts TensorFlow down there in all the measures it tested for. I can't ignore the possibility that this criticism of TensorFlow from Facebook employees (while factually correct and constructive) might be driven by some competition and…

This benchmark basically shows that releasing TensorFlow with cudnn v2 backend support hurts - v2 is quite a bit slower than v3 (current) and v4 (upcoming). TF has announced that they will update to v4 support, which should help quite a bit - but when many hobbyists and researchers are developing on one or two GPUs performance on that scale is more important (for them) than infinite scalability.

It is not surprising that a tool developed and focused on `Google scale` work has some imperfections in a wildly different setting. The question is - will they (or some dedicated contributor) speed up a use case the business itself may not have a use for? My gut feeling is that they will, but these things usually don`t happen overnight. Torch, Theano, and Caffe have years of work put into them and have largely been focused on the one to two GPU case.

Post reply on HN