21–30 of 50 posts
Nvidia Titan V can do 110 TFLOPS, 12GB of 1.7 Gb/s Memory [1] and sells for 3,000$. TPU v2 does 180 TFLOPS, 64GB of 19.2Gb/s Memory [2]. That's a heck of a performance boost for a chip that's likely costing google way less than the nvidia flagship. [1] http://www.tomshardware.com/news/nvidia-titan-v-110-teraflop...
Great talk, with lots of new insights into what's happening at Google. I really think his point that ImageNet is the new Mnist now holds true. Even research labs should be buying DeepLearning11 servers (10 x 1080Ti) for $15k, and training large models in a reasonable amount of time. It may seem that Google are way ahead, but they are just doing synchronous SGD, and it was interesting to see the drop in prediction acc…
That "Learned Index Structures" makes it pretty clear that Karpathy was right in his widely criticized "Software 2.0" piece.
I haven't read that paper (Learned Index Structures), but things like gperf have existed for decades. Are these enhanced data structures dynamic, i.e. unlike gperf which is a static one, does it reoptimize as you insert new elements? In the case of the hash table, I assume it's using the model to compute the hash function.
"An ... approach to handling inserts is to build a delta-index. All inserts are kept in buffer and from time to time merged with a potential retraining of the model."
That "Learned Index Structures" makes it pretty clear that Karpathy was right in his widely criticized "Software 2.0" piece.
I haven't read that paper (Learned Index Structures), but things like gperf have existed for decades. Are these enhanced data structures dynamic, i.e. unlike gperf which is a static one, does it reoptimize as you insert new elements? In the case of the hash table, I assume it's using the model to compute the hash function.
Earlier quoted context omitted.
I haven't read that paper (Learned Index Structures), but things like gperf have existed for decades. Are these enhanced data structures dynamic, i.e. unlike gperf which is a static one, does it reoptimize as you insert new elements? In the case of the hash table, I assume it's using the model to compute the hash function.
This is not like gperf.
Was it filmed? If yes, when video will be available?
Nvidia Titan V can do 110 TFLOPS, 12GB of 1.7 Gb/s Memory [1] and sells for 3,000$. TPU v2 does 180 TFLOPS, 64GB of 19.2Gb/s Memory [2]. That's a heck of a performance boost for a chip that's likely costing google way less than the nvidia flagship. [1] http://www.tomshardware.com/news/nvidia-titan-v-110-teraflop...
It's not clear to me how programmable the tpu is. I'm sure it's great at convolutions and matrix multiplies. Can it do anything else?
It's an accelerator to run Tensorflow graphs, and TF graphs essentially are converted to matrix operations and convolutions.
Earlier quoted context omitted.
I haven't read that paper (Learned Index Structures), but things like gperf have existed for decades. Are these enhanced data structures dynamic, i.e. unlike gperf which is a static one, does it reoptimize as you insert new elements? In the case of the hash table, I assume it's using the model to compute the hash function.
No, it doesn't handle inserts. On the other hand, the paper writes: "An ... approach to handling inserts is to build a delta-index. All inserts are kept in buffer and from time to time merged with a potential retraining of the model."
Great talk, with lots of new insights into what's happening at Google. I really think his point that ImageNet is the new Mnist now holds true. Even research labs should be buying DeepLearning11 servers (10 x 1080Ti) for $15k, and training large models in a reasonable amount of time. It may seem that Google are way ahead, but they are just doing synchronous SGD, and it was interesting to see the drop in prediction acc…
ballpark how much would it cost to train ImageNet (ILSVRC) on a std deep CNN arch (VGG or inception) on AWS using a p2 or p3?
On a DL11 server, it will take about 60 hrs, and only cost you 15k upfront. The economics speak for themselves for fp32 training, at this moment in time.