Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Neural network chip built using memristors

arstechnica.com

Post reply on HN
Print view View original

51–60 of 65 posts

Re: Neural network chip built using memristors

#51

Post by sz4kerto »

modeless wrote: post #30
meric wrote: post #15

A neural network chip semi-conductor startup: http://brainchipinc.com/technology/ They "backdoor listed" on to an Australian mining company, share price went from 1 cent to 27 cents: https://www.google.com/finance?cid=11163357 Valued at $57m.

Man, why are all the silicon people so fixated on spiking nets? Maybe in 20 years once we figure out how the brain works they'll be great but if you built a convnet chip instead then it could be smashing records in real problems like speech recognition, translation, image identification, etc, today . Where are the convnet chip startups?

Partly because they want to do fundamental research, not R&D. This means they don't necessarily have a well specified set of problems where they'd need neural networks, but they want to understand biological neural nets. One approach in doing so is trying to stay as close to real neural features as possible.

Re: Neural network chip built using memristors

#52

Post by dkersten »

thangalin wrote: post #27
andyl wrote: post #4

"Even on a 30 nm process, it would be possible to place 25 million cells in a square centimeter, with 10,000 synapses on each cell. And all that would dissipate about a Watt." Wow - seems like a lot. Human brain by comparison (sourced by google): - 12 watts - 100 billion neurons - 1000 trillion connections Computing with memsisters is going to be very interesting.

The human brain has between 100 and 500 trillion synapses and consumes a lowly 12 watts. (In contrast, a 12.6 megawatt supercomputer, in 2013, took 40 minutes to simulate one second of biological brain activity.) The article cites 250 billion synapses per watt. For the same 12 watts as a human brain eats up, a set of memristors could simulate three trillion synapses. A cat, in comparison, has 10 trillion. To get 100…

"In contrast, a 12.6 megawatt supercomputer, in 2013, took 40 minutes to simulate one second of biological brain activity."

Not really a fair comparison. That's like simulating one processor with another and complaining that 1 second of activity took 40 minutes to simulate. If we can implement the NN directly, rather than simulate it, we should expect a much smaller performance gap.

If your calculations are correct, that doesn't sound so bad. If we could implement a brain-sized (by number of neurons) neural network in the area as half of a medium Domino's pizza, that seems like a damn good achievement to me!

Re: Neural network chip built using memristors

#54

Post by jraines »

simi_ wrote: post #46
themeek wrote: post #11

This is one of the exotic devices in DARPA's UPSIDE competition for exascale computing. This initiative seeks to find non-state (non-transistor) based approaches to computation: exploitation of nanoscale response properties of discrete components to perform some restricted, non-binary, forms of computation. Essentially, exotic ways to abuse silicon lithography to get analog computation. The idea, and this can be seen…

I find it disturbing how everyone seems relatively unfazed by DARPA's intended use of this technology.

People have, I think, internalized the logic of arms races. Someone's going to have [scary technology X], so why shouldn't we have it first/best? (Sure, there are answers to that, but none that I think most people would stick to when push comes to shove).

Re: Neural network chip built using memristors

#55

Post by samatman »

sz4kerto wrote: post #50
thangalin wrote: post #27

Earlier quoted context omitted.

The human brain has between 100 and 500 trillion synapses and consumes a lowly 12 watts. (In contrast, a 12.6 megawatt supercomputer, in 2013, took 40 minutes to simulate one second of biological brain activity.) The article cites 250 billion synapses per watt. For the same 12 watts as a human brain eats up, a set of memristors could simulate three trillion synapses. A cat, in comparison, has 10 trillion. To get 100…

"In contrast, a 12.6 megawatt supercomputer, in 2013, took 40 minutes to simulate one second of biological brain activity." Just as a note: we are not sure whether that large computer really simulated brain activity or not. The tricky thing in brain research is that we have practically zero* idea about what matters and what can be omitted from the simulation. (For example, the glia cells seem to be important -- until…

As I never tire of quipping, if you can't get it high on neurotransmitters, it isn't a neural network.

Re: Neural network chip built using memristors

#56

Post by abstractbeliefs »

thangalin wrote: post #27
andyl wrote: post #4

"Even on a 30 nm process, it would be possible to place 25 million cells in a square centimeter, with 10,000 synapses on each cell. And all that would dissipate about a Watt." Wow - seems like a lot. Human brain by comparison (sourced by google): - 12 watts - 100 billion neurons - 1000 trillion connections Computing with memsisters is going to be very interesting.

The human brain has between 100 and 500 trillion synapses and consumes a lowly 12 watts. (In contrast, a 12.6 megawatt supercomputer, in 2013, took 40 minutes to simulate one second of biological brain activity.) The article cites 250 billion synapses per watt. For the same 12 watts as a human brain eats up, a set of memristors could simulate three trillion synapses. A cat, in comparison, has 10 trillion. To get 100…

Remember though that the brain seems to be a rather modular structure. While the total area is quite big, we could probably get a lot of mileage out of much smaller subsystems taking care of only, say, visual pattern recognition, or speech analysis, in much smaller dies.

Re: Neural network chip built using memristors

#57

Post by voteforchange »

abstractbeliefs wrote: post #56
thangalin wrote: post #27

Earlier quoted context omitted.

The human brain has between 100 and 500 trillion synapses and consumes a lowly 12 watts. (In contrast, a 12.6 megawatt supercomputer, in 2013, took 40 minutes to simulate one second of biological brain activity.) The article cites 250 billion synapses per watt. For the same 12 watts as a human brain eats up, a set of memristors could simulate three trillion synapses. A cat, in comparison, has 10 trillion. To get 100…

Remember though that the brain seems to be a rather modular structure. While the total area is quite big, we could probably get a lot of mileage out of much smaller subsystems taking care of only, say, visual pattern recognition, or speech analysis, in much smaller dies.

Someone should read Catherine Malibou and Francisco Varela :-)

Neuroscience is already bored with not only subsystems, the holistic nature of the brain, but also the notion of the brain as the body's sole processor of cognition.

Re: Neural network chip built using memristors

#58

Post by FD3SA »

Synaesthesia wrote: post #45
chm wrote: post #29

Earlier quoted context omitted.

How exactly is a memristor not a state device? And about journalistic coverage... you seem to be knowledgeable about these programs, so there's an opportunity for you :)

A lot of it is shrouded in secrecy I'm afraid, unless you're doing the research. I'm also very interested in memristors, I think it's a quantum leap forward for computing, in many respects. But there's very little information one can get out there. Would love to know where I can find out more.

The top comment mentions DARPA research, but for those who are actually interested in the theory behind the memristor and its potential applications, here [1] is an excellent resource by the man who discovered the first physical memristor.

Furthermore, there is a huge amount of open access journal articles and wikipedia material on memristors. DARPA is nowhere close to the cutting edge, SK Hynix is already tooling ReRAM fabs for mass production.

1. https://www.youtube.com/watch?v=bKGhvKyjgLY

Re: Neural network chip built using memristors

#59

Post by p1esk »

sz4kerto wrote: post #51
modeless wrote: post #30

Earlier quoted context omitted.

Man, why are all the silicon people so fixated on spiking nets? Maybe in 20 years once we figure out how the brain works they'll be great but if you built a convnet chip instead then it could be smashing records in real problems like speech recognition, translation, image identification, etc, today . Where are the convnet chip startups?

Partly because they want to do fundamental research, not R&D. This means they don't necessarily have a well specified set of problems where they'd need neural networks, but they want to understand biological neural nets. One approach in doing so is trying to stay as close to real neural features as possible.

Actually the reason (at least in this paper) is that spiking nets are much more power efficient.

Re: Neural network chip built using memristors

#60

Post by fapjacks »

themeek wrote: post #11

This is one of the exotic devices in DARPA's UPSIDE competition for exascale computing. This initiative seeks to find non-state (non-transistor) based approaches to computation: exploitation of nanoscale response properties of discrete components to perform some restricted, non-binary, forms of computation. Essentially, exotic ways to abuse silicon lithography to get analog computation. The idea, and this can be seen…

You know, I think it's pretty funny (with a heaping helping of schadenfreude) that the first graphic in the first link shows some completely arbitrary "DoD Sensing Requirements" compared to actual processor capability. How exactly does one bridge the gap between our current processor capabilities and fucking omniscience? I guess DARPA will try to find out.
Post reply on HN