Live data from Hacker News

Neural network chip built using memristors

arstechnica.com

41–50 of 65 posts

Re: Neural network chip built using memristors

#41
post #35

The technology sounds very promising but if the goal is to simulate the brain, the ANN models we have today are inadequate. Current evidence suggests that it needs to incorporate dendritic dynamics and , soon, molecular computation.

Is the goal simulation or functional equivalence?

For example, to simulate a horse, is it necessary to create legs, or is it okay to build a road and use wheels?

Re: Neural network chip built using memristors

#42
post #25

Earlier quoted context omitted.

I've previous EE experience, but I study CS. I'm open for project suggestions and would love to do research in self-assembly for mass fabrication and study/develop AI Models.

Well, the obvious project suggestion is to read about challenges of building a larger crossbar, then work on overcoming those challenges. Literature list is provided in the paper. This type of work all about mass fabrication, but has nothing to do with AI models. Which direction you want to go?

I think given my background I'm a better match for AI, than for mass fabrication. That's what I'd really enjoy working on.

Re: Neural network chip built using memristors

#43
post #9
post #4

"Even on a 30 nm process, it would be possible to place 25 million cells in a square centimeter, with 10,000 synapses on each cell. And all that would dissipate about a Watt." Wow - seems like a lot. Human brain by comparison (sourced by google): - 12 watts - 100 billion neurons - 1000 trillion connections Computing with memsisters is going to be very interesting.

put that on a 20cm x 30cm surface (laptop) ... and you have 25m x 600 = 15bn cells, 150 trillion synapses using 600W. If they can ever fabricate such a thing ... those neural networks are going to compute some scary stuff!

> If they can ever fabricate such a thing ... those neural networks are going to compute some scary stuff!

Do we have to pay extra for Austrian accent?

Re: Neural network chip built using memristors

#44
post #4

"Even on a 30 nm process, it would be possible to place 25 million cells in a square centimeter, with 10,000 synapses on each cell. And all that would dissipate about a Watt." Wow - seems like a lot. Human brain by comparison (sourced by google): - 12 watts - 100 billion neurons - 1000 trillion connections Computing with memsisters is going to be very interesting.

The human brain has between 100 and 500 trillion synapses and consumes a lowly 12 watts. (In contrast, a 12.6 megawatt supercomputer, in 2013, took 40 minutes to simulate one second of biological brain activity.) The article cites 250 billion synapses per watt. For the same 12 watts as a human brain eats up, a set of memristors could simulate three trillion synapses. A cat, in comparison, has 10 trillion. To get 100…

Brain synapse response time is very slow. Depending on type it is in order of milliseconds.

Re: Neural network chip built using memristors

#45
post #29
post #11

This is one of the exotic devices in DARPA's UPSIDE competition for exascale computing. This initiative seeks to find non-state (non-transistor) based approaches to computation: exploitation of nanoscale response properties of discrete components to perform some restricted, non-binary, forms of computation. Essentially, exotic ways to abuse silicon lithography to get analog computation. The idea, and this can be seen…

How exactly is a memristor not a state device? And about journalistic coverage... you seem to be knowledgeable about these programs, so there's an opportunity for you :)

A lot of it is shrouded in secrecy I'm afraid, unless you're doing the research. I'm also very interested in memristors, I think it's a quantum leap forward for computing, in many respects. But there's very little information one can get out there. Would love to know where I can find out more.

Re: Neural network chip built using memristors

#46
post #11

This is one of the exotic devices in DARPA's UPSIDE competition for exascale computing. This initiative seeks to find non-state (non-transistor) based approaches to computation: exploitation of nanoscale response properties of discrete components to perform some restricted, non-binary, forms of computation. Essentially, exotic ways to abuse silicon lithography to get analog computation. The idea, and this can be seen…

I find it disturbing how everyone seems relatively unfazed by DARPA's intended use of this technology.

Re: Neural network chip built using memristors

#47
post #34

Earlier quoted context omitted.

The human brain has between 100 and 500 trillion synapses and consumes a lowly 12 watts. (In contrast, a 12.6 megawatt supercomputer, in 2013, took 40 minutes to simulate one second of biological brain activity.) The article cites 250 billion synapses per watt. For the same 12 watts as a human brain eats up, a set of memristors could simulate three trillion synapses. A cat, in comparison, has 10 trillion. To get 100…

The thought of a drone with the intelligence of a cat is a scary thought... The numbers are interesting, though. 400 square centimetres sounds to me to be in the ballpark of a human brain (accounting for several layers).

> The thought of a drone with the intelligence of a cat is a scary thought...

And I immediately imagined drones going around the landing gears of bigger planes demanding their attention...

Re: Neural network chip built using memristors

#48
post #44

Earlier quoted context omitted.

The human brain has between 100 and 500 trillion synapses and consumes a lowly 12 watts. (In contrast, a 12.6 megawatt supercomputer, in 2013, took 40 minutes to simulate one second of biological brain activity.) The article cites 250 billion synapses per watt. For the same 12 watts as a human brain eats up, a set of memristors could simulate three trillion synapses. A cat, in comparison, has 10 trillion. To get 100…

Brain synapse response time is very slow. Depending on type it is in order of milliseconds.

I guess that's part of why such a large part of our thinking is subconscious parallel processing: as a workaround for this technical limitation.

Re: Neural network chip built using memristors

#49
post #35

The technology sounds very promising but if the goal is to simulate the brain, the ANN models we have today are inadequate. Current evidence suggests that it needs to incorporate dendritic dynamics and , soon, molecular computation.

Is the goal simulation or functional equivalence? For example, to simulate a horse, is it necessary to create legs, or is it okay to build a road and use wheels?

possibly, we don't know yet, but ANNs are not isomorphic to real neurons. If we are talking about a bottom-up approach to intelligence, we have to opt for realistic simulation. If, on the other hand we knew what intelligence is, we can simulate it any way we like.

Re: Neural network chip built using memristors

#50
post #4

"Even on a 30 nm process, it would be possible to place 25 million cells in a square centimeter, with 10,000 synapses on each cell. And all that would dissipate about a Watt." Wow - seems like a lot. Human brain by comparison (sourced by google): - 12 watts - 100 billion neurons - 1000 trillion connections Computing with memsisters is going to be very interesting.

The human brain has between 100 and 500 trillion synapses and consumes a lowly 12 watts. (In contrast, a 12.6 megawatt supercomputer, in 2013, took 40 minutes to simulate one second of biological brain activity.) The article cites 250 billion synapses per watt. For the same 12 watts as a human brain eats up, a set of memristors could simulate three trillion synapses. A cat, in comparison, has 10 trillion. To get 100…

"In contrast, a 12.6 megawatt supercomputer, in 2013, took 40 minutes to simulate one second of biological brain activity."

Just as a note: we are not sure whether that large computer really simulated brain activity or not. The tricky thing in brain research is that we have practically zero* idea about what matters and what can be omitted from the simulation. (For example, the glia cells seem to be important -- until recently, we have disregarded their role.)

So at this point even we had an infinitely big computer we could not simulate the brain properly because we don't know what exactly to simulate.

*zero means that there's much more we don't know than what we know.

Post reply on HN