Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Researchers make a “diffusive” memristor that emulates how a real synapse work

nanotechweb.org

Post reply on HN
Print view View original

11–18 of 18 posts

Re: Researchers make a “diffusive” memristor that emulates how a real synapse work

#11

Post by cbennett »

pohl wrote: post #10

I was surprised today that the entry on Wikipedia still qualifies the memristor as "hypothetical", and says that "there are...some serious doubts as to whether the memristor can actually exist in physical reality". I thought maybe research was farther along than that. https://en.wikipedia.org/wiki/Memristor

In truth it is. To practitioners in the field building circuits or doing simulations with nanodevices, this is somewhat of a tiresome debate.

To give some background, Leon Chua made certain claims about a hypothetical fourth circuit element and these debates largely stem back to claims about circuit analysis and mathematics. Basically his models predict a perfect device which, to my knowledge , has not been experimentally realized (to the contrary of HP's claims).

However , the funny thing is it doesn't really matter. We don't need a perfect memristor to build interesting and useful nanoionic and nano-redox circuits performing non-linear computational tasks. As modelers though, we do need to be careful making ideal claims about eternal non-volatility and device life (of course). Many point out that to the contrary of being ideal, these devices are extremely variable and imperfect- which is true. Anything built using nanofab techniques at the academic level (excluding semi-con industrial processes) will be..

Btw, If you want more physics depth on this , I can recommend any paper or book by Waser. They are all good. http://eu.wiley.com/WileyCDA/WileyTitle/productCd-3527334173...

Edit: adding a link to the first book chapter of aforementioned book I found which is already rather good. https://application.wiley-vch.de/books/sample/3527334173_c01...

Re: Researchers make a “diffusive” memristor that emulates how a real synapse work

#12

Post by hodwik »

pohl wrote: post #10

I was surprised today that the entry on Wikipedia still qualifies the memristor as "hypothetical", and says that "there are...some serious doubts as to whether the memristor can actually exist in physical reality". I thought maybe research was farther along than that. https://en.wikipedia.org/wiki/Memristor

I was confused by that as well. I was under the impression HP made working memristors years ago.

Re: Researchers make a “diffusive” memristor that emulates how a real synapse work

#13

Post by maxjohansen »

cbennett wrote: post #8
adrusi wrote: post #7

Earlier quoted context omitted.

Could a network be trained in software on powerful expensive hardware and then programmed onto some kind of neural FPGA that uses these memresistors to be used in power/space constrained systems?

Short answer: yes. Basically there are two options for future ReRAM (memristive) learning system: in-situ/on-chip learning , in which all learning rules are locally derived and enforced, and ex-situ learning systems in which we do what you suggested- import weights from more computationally/power expensive substrates. there is probably abundant promise in both approaches moving forward. I recommend looking at some re…

I would love to hear more about your research, please try to dig up the links.

Re: Researchers make a “diffusive” memristor that emulates how a real synapse work

#14

Post by cbennett »

maxjohansen wrote: post #13
cbennett wrote: post #8

Earlier quoted context omitted.

Short answer: yes. Basically there are two options for future ReRAM (memristive) learning system: in-situ/on-chip learning , in which all learning rules are locally derived and enforced, and ex-situ learning systems in which we do what you suggested- import weights from more computationally/power expensive substrates. there is probably abundant promise in both approaches moving forward. I recommend looking at some re…

I would love to hear more about your research, please try to dig up the links.

Glad to hear you find it exciting. I do too.. its a really hot field at the moment and I mean that in the good, not bad way ;) Lots of groups working in parallel on somewhat orthogonal design and architecture issues, with a variety of different considered devices, but a common basis set is emerging ;)

So, here's the paper I mentioned above. I think this is very methodical and inventive and definitely one of the best yet at considering confluence of DNNs and memristive (ReRAM) devices. A quick search revealed this was already on HN. https://arxiv.org/abs/1603.07341

So, I already mentioned the iconic Strukov paper above and my own which is really quite similar to Strukov in learning strategy/philosophy, except for we used entirely chemical and 'slow' devices , which may be quite interesting for brain emulation. (remember the brain operates in the mS , or microsecond regime and not nanosecond).

Here's another article I just stumbled upon a few days ago but which looks quite promising and brings us into the territory of a more un-supervised /probabilistic algorithm for learning. http://www.nature.com/articles/ncomms12611

Re: Researchers make a “diffusive” memristor that emulates how a real synapse work

#15

Post by posterboy »

Karlozkiller wrote: post #5
posterboy wrote: post #4

Earlier quoted context omitted.

I saw pictures labeled extracted from live brain scans. How that was done I don't know, maybe alpha or beta waves. At least that's short term memory.

Brain scans are not as accurate as one might think. Most or all methods merely show activation of neurons or groups of neurons over a timescale. Typically you give a person, say a memory task, you look at which areas fire up as he tries to memorise a word list and from that you extrapolate whatever you can from it. I also believe working memory and short term memory are to some extent better understood than long term…

I didn't express clearly. They scanned the brain and recreated some part of what the person sees.

1. http://news.berkeley.edu/2011/09/22/brain-movies/

2. http://gallantlab.org/_downloads/2011a.Nishimoto.etal.pdf

Spoilers: It is fMRI, indeed.

I'm not sure if the scan is inaccurate or the representation within the brain, but I suppose it had to be more accurate than just a fuzzy area.

Re: Researchers make a “diffusive” memristor that emulates how a real synapse work

#16

Post by [deleted] »

Karlozkiller wrote: post #5
posterboy wrote: post #4

Earlier quoted context omitted.

I saw pictures labeled extracted from live brain scans. How that was done I don't know, maybe alpha or beta waves. At least that's short term memory.

Brain scans are not as accurate as one might think. Most or all methods merely show activation of neurons or groups of neurons over a timescale. Typically you give a person, say a memory task, you look at which areas fire up as he tries to memorise a word list and from that you extrapolate whatever you can from it. I also believe working memory and short term memory are to some extent better understood than long term…

[deleted]

Re: Researchers make a “diffusive” memristor that emulates how a real synapse work

#17

Post by zamalek »

cbennett wrote: post #8
adrusi wrote: post #7

Earlier quoted context omitted.

Could a network be trained in software on powerful expensive hardware and then programmed onto some kind of neural FPGA that uses these memresistors to be used in power/space constrained systems?

Short answer: yes. Basically there are two options for future ReRAM (memristive) learning system: in-situ/on-chip learning , in which all learning rules are locally derived and enforced, and ex-situ learning systems in which we do what you suggested- import weights from more computationally/power expensive substrates. there is probably abundant promise in both approaches moving forward. I recommend looking at some re…

For a simple multi-layer feed-forward network, couldn't you just just use "classical" components - seeing as the parameters of the network would never change (having been trained beforehand)? I.e. Would you actually need memristors?

Re: Researchers make a “diffusive” memristor that emulates how a real synapse work

#18

Post by p1esk »

cbennett wrote: post #14
maxjohansen wrote: post #13

Earlier quoted context omitted.

I would love to hear more about your research, please try to dig up the links.

Glad to hear you find it exciting. I do too.. its a really hot field at the moment and I mean that in the good, not bad way ;) Lots of groups working in parallel on somewhat orthogonal design and architecture issues, with a variety of different considered devices, but a common basis set is emerging ;) So, here's the paper I mentioned above. I think this is very methodical and inventive and definitely one of the best…

What do you think about "chip in the loop" approach for training, which was popular in the 90s for hardware NN implementations?
Post reply on HN