11–20 of 57 posts
From Google's perspective it is probably more about how TensorFlow scales out horizontally. If a researcher fires off a Borg run (or whatever they use now) and the job takes a few thousand CPUs, no problem, at least for research. They must have better optimization a for running in production, such as in place operations.
The responses seem to show that the way you implement things can make a big difference in runtime. Perhaps the scripts used for benchmarking can be further optimized?
That said, the lack of in-place operations might be surprising (although it has been said that they are coming)
From Google's perspective it is probably more about how TensorFlow scales out horizontally. If a researcher fires off a Borg run (or whatever they use now) and the job takes a few thousand CPUs, no problem, at least for research. They must have better optimization a for running in production, such as in place operations.
From Google's perspective it is probably more about how TensorFlow scales out horizontally. If a researcher fires off a Borg run (or whatever they use now) and the job takes a few thousand CPUs, no problem, at least for research. They must have better optimization a for running in production, such as in place operations.
From Google's perspective it is probably more about how TensorFlow scales out horizontally. If a researcher fires off a Borg run (or whatever they use now) and the job takes a few thousand CPUs, no problem, at least for research. They must have better optimization a for running in production, such as in place operations.
Interesting benchmarks. One hopefully constructive critique, if you say things go out of memory, it'd be really useful to know what your setup is. Maybe you've got a big array of massive GPUs or you're running it on a more normal consumer GPU+box.
Interesting benchmarks. One hopefully constructive critique, if you say things go out of memory, it'd be really useful to know what your setup is. Maybe you've got a big array of massive GPUs or you're running it on a more normal consumer GPU+box.
According to: https://github.com/soumith/convnet-benchmarks , it's an NVIDIA TitanX (12GB GPU memory), which is pretty much the top of the line GPU for training neural nets. From my limited experience, most deep learning in research - including much of the state of the art - is done on single GPUs. (My guess is that if your model doesn't fit in 12GB, your model has way too many parameters to practically train anyway)…
Until now, I've seen two responses to Google's TensorFlow from Facebook employees. Yann Le Cunn seemed to really challenge Jeff Dean about TensorFlow's scalability [1] and this benchmark puts TensorFlow down there in all the measures it tested for. I can't ignore the possibility that this criticism of TensorFlow from Facebook employees (while factually correct and constructive) might be driven by some competition and…