Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If memristors act like neurons, put them in neural networks

spectrum.ieee.org

Post reply on HN
Print view View original

11–20 of 40 posts

Re: If memristors act like neurons, put them in neural networks

#11

Post by amelius »

waste_monk wrote: post #5
theamk wrote: post #3

I am surprised people have high expectations from memristors. They are just another way to build an analog computer -- better for machine learning, worse for classical ODEs. But we have not used analog computers for 50 years, and for a good reason -- they are not reproducible, their accuracy is very process dependent and has a hard upper limit, and they are often tuned for a single function. Would people want a chip…

>Would people want a chip which is basically unpredictable -- the performance can vary up by tens of %, they have to be re-trained periodically to prevent data loss, and there is no way to load pre-trained network? I doubt it. Maybe there is an extremely narrow use case, but I do not see it in the mainstream devices. Human brains have the same problems and seem to be fairly popular.

True, but they're hard to do experiments with, which is useful if you're building such a brain.

Re: If memristors act like neurons, put them in neural networks

#12

Post by jokethrowaway »

amelius wrote: post #11
waste_monk wrote: post #5

Earlier quoted context omitted.

>Would people want a chip which is basically unpredictable -- the performance can vary up by tens of %, they have to be re-trained periodically to prevent data loss, and there is no way to load pre-trained network? I doubt it. Maybe there is an extremely narrow use case, but I do not see it in the mainstream devices. Human brains have the same problems and seem to be fairly popular.

True, but they're hard to do experiments with, which is useful if you're building such a brain.

If the trade-off between energy-cheap human-like processing and our computers is reliability, it's definitely worth doing. If we had unlimited human like processing, we would basically have unlimited slaves without affecting biological organism similar to us.

Not to mention, it's worth doing it just for having 100000 times cheaper machine learning.

Re: If memristors act like neurons, put them in neural networks

#13

Post by MauranKilom »

waste_monk wrote: post #5
theamk wrote: post #3

I am surprised people have high expectations from memristors. They are just another way to build an analog computer -- better for machine learning, worse for classical ODEs. But we have not used analog computers for 50 years, and for a good reason -- they are not reproducible, their accuracy is very process dependent and has a hard upper limit, and they are often tuned for a single function. Would people want a chip…

>Would people want a chip which is basically unpredictable -- the performance can vary up by tens of %, they have to be re-trained periodically to prevent data loss, and there is no way to load pre-trained network? I doubt it. Maybe there is an extremely narrow use case, but I do not see it in the mainstream devices. Human brains have the same problems and seem to be fairly popular.

Human brains are also ridiculously expensive to copy and distribute. Not to mention that you need an extremely loose definition of "copy".

Re: If memristors act like neurons, put them in neural networks

#15

Post by birdyrooster »

jokethrowaway wrote: post #12
amelius wrote: post #11

Earlier quoted context omitted.

True, but they're hard to do experiments with, which is useful if you're building such a brain.

If the trade-off between energy-cheap human-like processing and our computers is reliability, it's definitely worth doing. If we had unlimited human like processing, we would basically have unlimited slaves without affecting biological organism similar to us. Not to mention, it's worth doing it just for having 100000 times cheaper machine learning.

Oh boy I can’t wait for the unlimited slaves

Re: If memristors act like neurons, put them in neural networks

#16

Post by svantana »

theamk wrote: post #3

I am surprised people have high expectations from memristors. They are just another way to build an analog computer -- better for machine learning, worse for classical ODEs. But we have not used analog computers for 50 years, and for a good reason -- they are not reproducible, their accuracy is very process dependent and has a hard upper limit, and they are often tuned for a single function. Would people want a chip…

> Would people want a chip which is basically unpredictable

Judging by the prevalence of (pseudo-) random numbers in machine learning, I'd say yes. Reentrancy is a big plus, but not always a dealbreaker.

It's possible we might end up with a sort of left/right-brain setup, with a noisy analog hypothesis generator paired with a robust, logic-based evaluator/planner.

Re: If memristors act like neurons, put them in neural networks

#17

Post by eru »

svantana wrote: post #16
theamk wrote: post #3

I am surprised people have high expectations from memristors. They are just another way to build an analog computer -- better for machine learning, worse for classical ODEs. But we have not used analog computers for 50 years, and for a good reason -- they are not reproducible, their accuracy is very process dependent and has a hard upper limit, and they are often tuned for a single function. Would people want a chip…

> Would people want a chip which is basically unpredictable Judging by the prevalence of (pseudo-) random numbers in machine learning, I'd say yes. Reentrancy is a big plus, but not always a dealbreaker. It's possible we might end up with a sort of left/right-brain setup, with a noisy analog hypothesis generator paired with a robust, logic-based evaluator/planner.

Those pseudo-random numbers are still reproducable.

Re: If memristors act like neurons, put them in neural networks

#18

Post by svantana »

eru wrote: post #17
svantana wrote: post #16

Earlier quoted context omitted.

> Would people want a chip which is basically unpredictable Judging by the prevalence of (pseudo-) random numbers in machine learning, I'd say yes. Reentrancy is a big plus, but not always a dealbreaker. It's possible we might end up with a sort of left/right-brain setup, with a noisy analog hypothesis generator paired with a robust, logic-based evaluator/planner.

Those pseudo-random numbers are still reproducable.

Sorry, I meant reproducible, not reentrant.

Re: If memristors act like neurons, put them in neural networks

#19

Post by sesuximo »

TeMPOraL wrote: post #10
sgtnoodle wrote: post #8

Earlier quoted context omitted.

They aren't really directly comparable like you imply. A memristor is basically a resistor that changes resistance when you run current through it. An FPGA is a clever layout of many thousands of logic gates that can be programmed to form arbitrarily complex digital logic circuits. It's like comparing apples to skyscrapers because they're both associated with New York. Presumably, a memristor based neural network wou…

Correct me if I'm wrong, but isn't it a good metaphor to say that FPGAs are just ASIC emulators? I.e. much less efficient than an equivalent ASIC, but good for prototyping and something to use in production if you can't afford to manufacture your own chip (which is most low-volume use cases). Under this metaphor, memristor-based ASICs will obviously be more efficient than FPGAs emulating them, but that's predicated o…

Wow there... sometimes digital emulation of an analog circuit is faster than the analog circuit since you can solve for equilibria, easily repeat calculations, use well researched programming tools, etc

Re: If memristors act like neurons, put them in neural networks

#20

Post by danw1979 »

MauranKilom wrote: post #13
waste_monk wrote: post #5

Earlier quoted context omitted.

>Would people want a chip which is basically unpredictable -- the performance can vary up by tens of %, they have to be re-trained periodically to prevent data loss, and there is no way to load pre-trained network? I doubt it. Maybe there is an extremely narrow use case, but I do not see it in the mainstream devices. Human brains have the same problems and seem to be fairly popular.

Human brains are also ridiculously expensive to copy and distribute. Not to mention that you need an extremely loose definition of "copy".

Me and the wife made a couple and whilst they aren't cheap, I wouldn't say they're outrageously costly to make/run.

Not particularly reliable though... and yes, we seem to be having great difficulty copying the required information into them.

Post reply on HN