Hardware Architectures for Deep Neural Networks [pdf]
1–10 of 19 posts
Re: Hardware Architectures for Deep Neural Networks [pdf]
#2link seems broken!
Re: Hardware Architectures for Deep Neural Networks [pdf]
#3There is also a paper from same authors with about the same content:
Efficient Processing of Deep Neural Networks: A Tutorial and Survey Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, Joel Emer
Re: Hardware Architectures for Deep Neural Networks [pdf]
#4link seems broken!
Not broken for me - it just needs a while to load, the file is huge
Re: Hardware Architectures for Deep Neural Networks [pdf]
#5link seems broken!
It's a 32MB PDF, as it's image heavy, and takes a while to download from their servers.
Re: Hardware Architectures for Deep Neural Networks [pdf]
#6I've been wanting to know how electronics' power efficiency compares to biological neurons, and this paper gives a clue. The most efficient hardware it mentions is the GV100 "Tensor Core", at 400GFLOPS/W for FP16.
If a typical neuron requires 10^6 ATP per activation [1], and if it takes 30.5 kJ/mol to charge ATP [2], and if a typical neuron has 100 axons, each of which is contributing one FLOP, then I _think_ a human neuron is about 500 times as efficient as a GV100 [3], at 200,000 GFLOPS per watt.
[1] https://www.extremetech.com/extreme/185984-the-human-brains-...
[2] https://en.wikipedia.org/wiki/Adenosine_triphosphate
[3]
Neuron:
10E6 ATP = 1 activation = 100 FLOP
30.5 kJ = 1 mole ATP = 6E23 ATP
1 kJ = 0.28 Wh
...
2E5 GFLOPS = 1 W
GV100 "Tensor Core":
4E2 GFLOPS = 1 WRe: Hardware Architectures for Deep Neural Networks [pdf]
#7I've been wanting to know how electronics' power efficiency compares to biological neurons, and this paper gives a clue. The most efficient hardware it mentions is the GV100 "Tensor Core", at 400GFLOPS/W for FP16. If a typical neuron requires 10^6 ATP per activation [1], and if it takes 30.5 kJ/mol to charge ATP [2], and if a typical neuron has 100 axons, each of which is contributing one FLOP, then I _think_ a human…
Keep in mind the energy costs of the supporting hardware for that brain, though.
Re: Hardware Architectures for Deep Neural Networks [pdf]
#8link seems broken!
290 image-heavy pages. smh
Re: Hardware Architectures for Deep Neural Networks [pdf]
#9I've been wanting to know how electronics' power efficiency compares to biological neurons, and this paper gives a clue. The most efficient hardware it mentions is the GV100 "Tensor Core", at 400GFLOPS/W for FP16. If a typical neuron requires 10^6 ATP per activation [1], and if it takes 30.5 kJ/mol to charge ATP [2], and if a typical neuron has 100 axons, each of which is contributing one FLOP, then I _think_ a human…
Well, keep in mind that you should never really use the advertised number as your comparison point, because that's more the "highest that you'll ever see" rather than the "average reliable performance". Not to mention they assume everything magically is in the caches as far as memory is concerned, which is absolutely untrue.
Re: Hardware Architectures for Deep Neural Networks [pdf]
#10I've been wanting to know how electronics' power efficiency compares to biological neurons, and this paper gives a clue. The most efficient hardware it mentions is the GV100 "Tensor Core", at 400GFLOPS/W for FP16. If a typical neuron requires 10^6 ATP per activation [1], and if it takes 30.5 kJ/mol to charge ATP [2], and if a typical neuron has 100 axons, each of which is contributing one FLOP, then I _think_ a human…
[deleted]