Neural network chip built using memristors • Hacker News Board

Re: Neural network chip built using memristors

#61

Post by sushirain » Thu, May 07, 2015, 8:34 PM UTC

p1esk wrote: post #37

modeless wrote: post #32

Earlier quoted context omitted.

That paper does not implement training, only testing. Low-power testing is good to have e.g. for mobile applications, but it doesn't increase the capabilities of our algorithms. What we need to advance the field is faster training of larger nets. That's where the really interesting applications will be discovered. A convnet-optimized chip could clearly be much faster and more power efficient than a convnet running on…

Building a hardware convnet is only beneficial when you figured out the exact parameters of the network. Everything is hardwired. Therefore, it's useless if you want to experiment with lots of different parameters, tricks, or architectures to "advance the field". Moreover, building such a chip is an expensive and long process, and given how fast GPUs are improving, it's not clear that by the time you build it, it wil…

Maybe the brain can do one shot learning only after it has trained for years?

Re: Neural network chip built using memristors

#62

Post by p1esk » Thu, May 07, 2015, 9:25 PM UTC

sushirain wrote: post #61

p1esk wrote: post #37

Earlier quoted context omitted.

Building a hardware convnet is only beneficial when you figured out the exact parameters of the network. Everything is hardwired. Therefore, it's useless if you want to experiment with lots of different parameters, tricks, or architectures to "advance the field". Moreover, building such a chip is an expensive and long process, and given how fast GPUs are improving, it's not clear that by the time you build it, it wil…

Maybe the brain can do one shot learning only after it has trained for years?

Maybe. We want to reduce the amount of supervised training. I think that's what Hinton is trying to do with his capsules.

Re: Neural network chip built using memristors

#63

Post by themeek » Thu, May 07, 2015, 10:21 PM UTC

fapjacks wrote: post #60

themeek wrote: post #11

This is one of the exotic devices in DARPA's UPSIDE competition for exascale computing. This initiative seeks to find non-state (non-transistor) based approaches to computation: exploitation of nanoscale response properties of discrete components to perform some restricted, non-binary, forms of computation. Essentially, exotic ways to abuse silicon lithography to get analog computation. The idea, and this can be seen…

You know, I think it's pretty funny (with a heaping helping of schadenfreude) that the first graphic in the first link shows some completely arbitrary "DoD Sensing Requirements" compared to actual processor capability. How exactly does one bridge the gap between our current processor capabilities and fucking omniscience ? I guess DARPA will try to find out.

They intend to bridge the gap by forfeiting the assumption that transistors and discrete logic need be the atomic units of computation. They give up discretization and call for exotic modes of computation that can achieve reliable, but non binary, behaviors. Chaining these new units together, the idea is to build 'inference modules' - small clusters capable of performing statistical induction and aduction - these modules than able to be chained to perform higher level logic like, for example, detecting shapes and features in input video for tracking.

Re: Neural network chip built using memristors

#64

Post by chm » Fri, May 08, 2015, 4:19 AM UTC

Synaesthesia wrote: post #45

chm wrote: post #29

Earlier quoted context omitted.

How exactly is a memristor not a state device? And about journalistic coverage... you seem to be knowledgeable about these programs, so there's an opportunity for you :)

A lot of it is shrouded in secrecy I'm afraid, unless you're doing the research. I'm also very interested in memristors, I think it's a quantum leap forward for computing, in many respects. But there's very little information one can get out there. Would love to know where I can find out more.

I suggest you start with the Nature paper from the guys at HP[1]. Then move on to Leon Chua's original and later papers.

[1]: http://www.nature.com/nature/journal/v453/n7191/full/nature0...

Re: Neural network chip built using memristors

#65

Post by Lozi » Sat, May 09, 2015, 5:33 PM UTC

The general theory of memristors, meminductors, memcapacitors of any order (first order memristor is the genuine one invented in 1971 by Professor Leon Chua, however he generalized recently his discover to second, third order memristor, etc.)is published in a paper I wrote with him on september 2014 in International Journal of Bifurcation and Chaos. One can download it freely from the site https://www.researchgate.net/publication/261676241_MEMFRACTA...