Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If memristors act like neurons, put them in neural networks

spectrum.ieee.org

Post reply on HN
Print view View original

31–40 of 40 posts

Re: If memristors act like neurons, put them in neural networks

#31

Post by JoeAltmaier »

ddingus wrote: post #30
JoeAltmaier wrote: post #28

Earlier quoted context omitted.

Is the better alternative, to create actual slaves? Intelligent feeling beings that we put governors on, like a Thrall shock-collar from Star Trek, so they don't get to uppity? Because that's a horror show of its own right there.

[Goes with morbid flow here] Perhaps! If we get it right, and they are in fact beings, there may not be better alternatives. May always end in conflict. I say that because a core aspect of beings, is that they "be", and that boils down to agency of some kind. Negative reinforcement runs in conflict to all that and warrants a response, from the oppressed beings point of view. Us being creators will only go so far. Joh…

Ok that's not a shock collar, but creepily similar to keeping folks drugged into compliance. Also an ethical hole.

I'm not sure what the solution is. Intelligent but not self-aware?

Re: If memristors act like neurons, put them in neural networks

#32

Post by ddingus »

JoeAltmaier wrote: post #31
ddingus wrote: post #30

Earlier quoted context omitted.

[Goes with morbid flow here] Perhaps! If we get it right, and they are in fact beings, there may not be better alternatives. May always end in conflict. I say that because a core aspect of beings, is that they "be", and that boils down to agency of some kind. Negative reinforcement runs in conflict to all that and warrants a response, from the oppressed beings point of view. Us being creators will only go so far. Joh…

Ok that's not a shock collar, but creepily similar to keeping folks drugged into compliance. Also an ethical hole. I'm not sure what the solution is. Intelligent but not self-aware?

Yeah the whole thing is a mess. I don't think there is a solution.

Things that are intelligent in some fashion but not self-aware are machines.

Edit: We will end up making really good machines, and they will present us with the illusion of agency. That's all a good thing. I want agents for a variety of things. Most of us do, and or would benefit from them.

But, none of that actually requires we make other beings. Once we do that, they are beings! They will have agency and all the other stuff we associate with sentience. And the way I see all that is they are peers. They may be simpler than us, like the animals are, or more than us, with obvious and to many, concerning implications.

When we advance machines, no shocks, extreme pleasure, type means and methods are required. And, the price of that is something almost aware, but lacking agency.

Everything pivots on agency.

Should we create something having it, the idea is it can do hard work for us because it will be capable of it like we are.

Otherwise, we have to do all the hard work and end up with something able to do more work, but the investment cost is high. Running costs are much lower. New hard work = another significant investment.

I, nor anyone I believe, can tell us how agency, the self, being aware, and all that intertwine to present as intelligence, as a being of some sort. Answers come slow, and I fear the tests needed to get at them more quickly are ethics bombs all over the place.

We want to make that happen somehow on the assumption we will get a huge return and that because of the potential we know intelligence can bring to the table.

It all seems an awful lot like, "wants cake to eat it too" at this point in time.

Re: If memristors act like neurons, put them in neural networks

#33

Post by tartoran »

svantana wrote: post #16
theamk wrote: post #3

I am surprised people have high expectations from memristors. They are just another way to build an analog computer -- better for machine learning, worse for classical ODEs. But we have not used analog computers for 50 years, and for a good reason -- they are not reproducible, their accuracy is very process dependent and has a hard upper limit, and they are often tuned for a single function. Would people want a chip…

> Would people want a chip which is basically unpredictable Judging by the prevalence of (pseudo-) random numbers in machine learning, I'd say yes. Reentrancy is a big plus, but not always a dealbreaker. It's possible we might end up with a sort of left/right-brain setup, with a noisy analog hypothesis generator paired with a robust, logic-based evaluator/planner.

> It's possible we might end up with a sort of left/right-brain setup, with a noisy analog hypothesis generator paired with a robust, logic-based evaluator/planner.

That sounds very interesting to explore, definitely would teach us something new.

Re: If memristors act like neurons, put them in neural networks

#34

Post by amelius »

JoeAltmaier wrote: post #28
ddingus wrote: post #27

Earlier quoted context omitted.

Don't worry, we will take short cuts, because greed and ignorance, leaving them with agency we will not have accounted for. Should be exciting.

Is the better alternative, to create actual slaves? Intelligent feeling beings that we put governors on, like a Thrall shock-collar from Star Trek, so they don't get to uppity? Because that's a horror show of its own right there.

Perhaps we can give the slaves feelings of profound happiness when they obey us. Requires some immense scientific and/or philosophical advances though.

Re: If memristors act like neurons, put them in neural networks

#35

Post by avs733 »

danw1979 wrote: post #20
MauranKilom wrote: post #13

Earlier quoted context omitted.

Human brains are also ridiculously expensive to copy and distribute. Not to mention that you need an extremely loose definition of "copy".

Me and the wife made a couple and whilst they aren't cheap, I wouldn't say they're outrageously costly to make/run. Not particularly reliable though... and yes, we seem to be having great difficulty copying the required information into them.

I'm in the first trimester of the manufacturing process, excited but also having some sticker shock at day care options.

Never built something so initially unstable that ends up Turing complete before.

Re: If memristors act like neurons, put them in neural networks

#36

Post by 3JPLW »

I encourage folks to actually read the linked article instead of basing their commentary on the shoddy title.

https://www.nature.com/articles/s41928-020-00523-3

Abstract:

> Resistive memory technologies could be used to create intelligent systems that learn locally at the edge. However, current approaches typically use learning algorithms that cannot be reconciled with the intrinsic non-idealities of resistive memory, particularly cycle-to-cycle variability. Here, we report a machine learning scheme that exploits memristor variability to implement Markov chain Monte Carlo sampling in a fabricated array of 16,384 devices configured as a Bayesian machine learning model. We apply the approach experimentally to carry out malignant tissue recognition and heart arrhythmia detection tasks, and, using a calibrated simulator, address the cartpole reinforcement learning task. Our approach demonstrates robustness to device degradation at ten million endurance cycles, and, based on circuit and system-level simulations, the total energy required to train the models is estimated to be on the order of microjoules, which is notably lower than in complementary metal– oxide–semiconductor (CMOS)-based approaches.

Re: If memristors act like neurons, put them in neural networks

#37

Post by theamk »

Grambo wrote: post #25
theamk wrote: post #4

Earlier quoted context omitted.

What do you think of memristors vs FPGAs? They have many similarities -- they both redrew the current computer architectures, they integrate memory and computing, they can have randomness built-in, and they both take less power than mainstream GPUs. What does memristor provide that specially designed "neural FPGA" can not?

My understanding of the use of FPGAs and ASICs that are used to speed up neural networks (such as those in phones) is that they are simply designed to do the types of calculations used for NNs more quickly (matrix operations) and generally at a reduced level of precision. This is very different from a memristor approach where the structure of the network itself would be represented in the silicon. I also think it's u…

"the structure of the network itself would be represented in the silicon" -- ASICs then? perhaps even hybrid analog/digital one, where fixed coefficients are stored in digital memory, while input data is analog.

I believe there is a great value in being able to "snapshot" the state and later load exactly the same state into millions of devices. And I cannot see how this will easily work with memristors.

Re: If memristors act like neurons, put them in neural networks

#38

Post by YeGoblynQueenne »

>> The devices could also work well within neural networks, which are machine learning systems that use synthetic versions of synapses and neurons to mimic the process of learning in the human brain.

Yann LeCun disagrees:

IEEE Spectrum: We read about Deep Learning in the news a lot these days. What’s your least favorite definition of the term that you see in these stories?

Yann LeCun: My least favorite description is, “It works just like the brain.” I don’t like people saying this because, while Deep Learning gets an inspiration from biology, it’s very, very far from what the brain actually does. And describing it like the brain gives a bit of the aura of magic to it, which is dangerous. It leads to hype; people claim things that are not true. AI has gone through a number of AI winters because people claimed things they couldn’t deliver.

https://spectrum.ieee.org/automaton/artificial-intelligence/...

Post reply on HN