Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If memristors act like neurons, put them in neural networks

spectrum.ieee.org

Post reply on HN
Print view View original

21–30 of 40 posts

Re: If memristors act like neurons, put them in neural networks

#21

Post by avs733 »

amelius wrote: post #11
waste_monk wrote: post #5

Earlier quoted context omitted.

>Would people want a chip which is basically unpredictable -- the performance can vary up by tens of %, they have to be re-trained periodically to prevent data loss, and there is no way to load pre-trained network? I doubt it. Maybe there is an extremely narrow use case, but I do not see it in the mainstream devices. Human brains have the same problems and seem to be fairly popular.

True, but they're hard to do experiments with, which is useful if you're building such a brain.

Not really...

Psychology does this all the time.

Outcomes can be described as distributions as much as they can binaries. It just takes a change in mindset.

Re: If memristors act like neurons, put them in neural networks

#22

Post by Datenstrom »

MauranKilom wrote: post #13
waste_monk wrote: post #5

Earlier quoted context omitted.

>Would people want a chip which is basically unpredictable -- the performance can vary up by tens of %, they have to be re-trained periodically to prevent data loss, and there is no way to load pre-trained network? I doubt it. Maybe there is an extremely narrow use case, but I do not see it in the mainstream devices. Human brains have the same problems and seem to be fairly popular.

Human brains are also ridiculously expensive to copy and distribute. Not to mention that you need an extremely loose definition of "copy".

> Not to mention that you need an extremely loose definition of "copy".

Makes me wonder, with a nearly perfect simulation for a reinforcement learning agent and a human brain or equivalent architecture, would you be able to make exact copies. Assuming the PRNGs and other settings were deterministic. Now that would be an interesting and controversial experiment.

Re: If memristors act like neurons, put them in neural networks

#23

Post by Veedrac »

waste_monk wrote: post #5
theamk wrote: post #3

I am surprised people have high expectations from memristors. They are just another way to build an analog computer -- better for machine learning, worse for classical ODEs. But we have not used analog computers for 50 years, and for a good reason -- they are not reproducible, their accuracy is very process dependent and has a hard upper limit, and they are often tuned for a single function. Would people want a chip…

>Would people want a chip which is basically unpredictable -- the performance can vary up by tens of %, they have to be re-trained periodically to prevent data loss, and there is no way to load pre-trained network? I doubt it. Maybe there is an extremely narrow use case, but I do not see it in the mainstream devices. Human brains have the same problems and seem to be fairly popular.

Human brains try really hard not to be analogue.

Re: If memristors act like neurons, put them in neural networks

#24

Post by MichaelZuo »

birdyrooster wrote: post #15
jokethrowaway wrote: post #12

Earlier quoted context omitted.

If the trade-off between energy-cheap human-like processing and our computers is reliability, it's definitely worth doing. If we had unlimited human like processing, we would basically have unlimited slaves without affecting biological organism similar to us. Not to mention, it's worth doing it just for having 100000 times cheaper machine learning.

Oh boy I can’t wait for the unlimited slaves

Well bounded by the expanding co-moving light cone of mass-energy, barring FTL travel.

Re: If memristors act like neurons, put them in neural networks

#25

Post by Grambo »

theamk wrote: post #4
peatmoss wrote: post #2

Memristors are the technology that seemed poised to usher in a new era of computing. The promise of being able to redraw the current computer architectural hierarchies is tantalizing. If machine learning applications are what finally get memristors out into the world, I wish them godspeed.

What do you think of memristors vs FPGAs? They have many similarities -- they both redrew the current computer architectures, they integrate memory and computing, they can have randomness built-in, and they both take less power than mainstream GPUs. What does memristor provide that specially designed "neural FPGA" can not?

My understanding of the use of FPGAs and ASICs that are used to speed up neural networks (such as those in phones) is that they are simply designed to do the types of calculations used for NNs more quickly (matrix operations) and generally at a reduced level of precision. This is very different from a memristor approach where the structure of the network itself would be represented in the silicon. I also think it's unfair to compare the two because it took decades of work to get CMOS transistors to where they are today. I imagine that once commercial applications for memristors appear many optimizations/improvements will present themselves.

Re: If memristors act like neurons, put them in neural networks

#26

Post by ddingus »

waste_monk wrote: post #5
theamk wrote: post #3

I am surprised people have high expectations from memristors. They are just another way to build an analog computer -- better for machine learning, worse for classical ODEs. But we have not used analog computers for 50 years, and for a good reason -- they are not reproducible, their accuracy is very process dependent and has a hard upper limit, and they are often tuned for a single function. Would people want a chip…

>Would people want a chip which is basically unpredictable -- the performance can vary up by tens of %, they have to be re-trained periodically to prevent data loss, and there is no way to load pre-trained network? I doubt it. Maybe there is an extremely narrow use case, but I do not see it in the mainstream devices. Human brains have the same problems and seem to be fairly popular.

My thoughts exactly.

People want what is in our heads. Which is it?

Some idealized, and by nature limited sub set or idea, or the real deal?

Re: If memristors act like neurons, put them in neural networks

#27

Post by ddingus »

birdyrooster wrote: post #15
jokethrowaway wrote: post #12

Earlier quoted context omitted.

If the trade-off between energy-cheap human-like processing and our computers is reliability, it's definitely worth doing. If we had unlimited human like processing, we would basically have unlimited slaves without affecting biological organism similar to us. Not to mention, it's worth doing it just for having 100000 times cheaper machine learning.

Oh boy I can’t wait for the unlimited slaves

Don't worry, we will take short cuts, because greed and ignorance, leaving them with agency we will not have accounted for.

Should be exciting.

Re: If memristors act like neurons, put them in neural networks

#28

Post by JoeAltmaier »

ddingus wrote: post #27
birdyrooster wrote: post #15

Earlier quoted context omitted.

Oh boy I can’t wait for the unlimited slaves

Don't worry, we will take short cuts, because greed and ignorance, leaving them with agency we will not have accounted for. Should be exciting.

Is the better alternative, to create actual slaves? Intelligent feeling beings that we put governors on, like a Thrall shock-collar from Star Trek, so they don't get to uppity?

Because that's a horror show of its own right there.

Re: If memristors act like neurons, put them in neural networks

#29

Post by tartoran »

waste_monk wrote: post #5
theamk wrote: post #3

I am surprised people have high expectations from memristors. They are just another way to build an analog computer -- better for machine learning, worse for classical ODEs. But we have not used analog computers for 50 years, and for a good reason -- they are not reproducible, their accuracy is very process dependent and has a hard upper limit, and they are often tuned for a single function. Would people want a chip…

>Would people want a chip which is basically unpredictable -- the performance can vary up by tens of %, they have to be re-trained periodically to prevent data loss, and there is no way to load pre-trained network? I doubt it. Maybe there is an extremely narrow use case, but I do not see it in the mainstream devices. Human brains have the same problems and seem to be fairly popular.

> Human brains have the same problems and seem to be fairly popular.

Not only that but I'm sure there's a usecase where they are more suitable, we just haven't found it.

Re: If memristors act like neurons, put them in neural networks

#30

Post by ddingus »

JoeAltmaier wrote: post #28
ddingus wrote: post #27

Earlier quoted context omitted.

Don't worry, we will take short cuts, because greed and ignorance, leaving them with agency we will not have accounted for. Should be exciting.

Is the better alternative, to create actual slaves? Intelligent feeling beings that we put governors on, like a Thrall shock-collar from Star Trek, so they don't get to uppity? Because that's a horror show of its own right there.

[Goes with morbid flow here]

Perhaps!

If we get it right, and they are in fact beings, there may not be better alternatives. May always end in conflict.

I say that because a core aspect of beings, is that they "be", and that boils down to agency of some kind.

Negative reinforcement runs in conflict to all that and warrants a response, from the oppressed beings point of view. Us being creators will only go so far.

John Varley proposed an interesting solution to this in his Gaea series. A slave was created by connecting all their pleasure centers to an area on their forehead. Any touch other than their own triggered pleasure so intense it is worth pretty much anything to experience.

Post reply on HN