Live data from Hacker News

Memristor Breakthrough: First Single Device to Act Like a Neuron

spectrum.ieee.org

71–80 of 112 posts

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#71
post #24

Earlier quoted context omitted.

We don't need to go fully analogue though, going asynchronous might already be a step up. In neural networks some groups do research into so-called spiking neural networks, which you could think of as electronic circuits operating asynchronously on binary signals. They are very energy efficient, but nobody really knows how to use them yet.

Clockless CPUs have been a theoretical idea for over a decade before the "neutral network" became a common phrase. Felt like a decade ago it was still a pipe dream. Resolving the asynchronous signals becomes a chore. The world feels stateful and ordered to humans. In the same way functional programming can be difficult, getting this right is very hard.

There had been purely asynchronous CPUs before [0]. Then automated clock gate insertion made it possible to approach the gains of asynchronous, but still designing in a synchronous paradigm.

[0] https://en.m.wikipedia.org/wiki/AMULET_microprocessor

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#73
What neural nets need in order to improve is lots of data, and especially simulated environments. We have had the advantage of billions of years of evolution, they start the same journey from scratch.

Human brain is not smart in and of itself - it learns everything from the environment, including fundamental concepts and reasoning. We are not just surrounded by nature, with all its glorious detail, but also by the human society and culture. Our environment is very complex and rich. That's something neural nets have to replicate some other way, and simulation is one.

It shows that just inventing a better artificial neuron does not meaningfully advance the problem of artificial intelligence. It's just one leg, it needs two legs to stand on.

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#74

Earlier quoted context omitted.

It's fascinating that none of the comments mention neural network applications. Seems like these people came a bit too early to the party.

We've left biomimicry in the dust a fair time ago - neurons do not work like relus after all (nor do they work like memristors), but relus are great at solving problems. None of our current NN applications would even want something that behaves like a neuron. You'd essentially have to start from scratch. It's not even clear if behaves-like-a-neuron is better or not to be worth the effort. (And that's disregarding the…

Do we even understand how a neuron works well enough to say that either thing is correct. I thought a relu was just the activation function that decides what weight to triggers the next part of the network with?

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#75

Earlier quoted context omitted.

It's fascinating that none of the comments mention neural network applications. Seems like these people came a bit too early to the party.

We've left biomimicry in the dust a fair time ago - neurons do not work like relus after all (nor do they work like memristors), but relus are great at solving problems. None of our current NN applications would even want something that behaves like a neuron. You'd essentially have to start from scratch. It's not even clear if behaves-like-a-neuron is better or not to be worth the effort. (And that's disregarding the…

I don't think Relus are the proplem. Relus are good, but artificial neural networks can work with sigmoid activation fonctions instead. The neurones in an ANN with sigmoids are somewhat close to what is described in this article.

The main problem is that the neurons in this article are not trainable (you can't update the weights dynamically) and not differentiable (you can't use the usual gradient-based training procedures to update the weights). They could be used for inference only though.

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#76
The news article mentions that the student on the paper has passed away. He died of suicide in his lab just a couple weeks after the paper was submitted to Nature [0]. Grad school is hard for a lot of people. Find someone to talk to if you're struggling.

[0] https://padailypost.com/2019/02/16/engineering-student-found...

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#77
post #16

This is really interesting work, but I feel like the way the article summarizes it is getting it wrong. First Single Device To Act Like a Neuron One thing that’s kept engineers from copying the brain’s power efficiency and quirky computational skill is the lack of an electronic device that can, all on its own, act like a neuron. It would take a special kind of device to do that, one whose behavior is more complex tha…

Modern NNs run like that because all we had was a hammer so we tried as hard as possible to make the NNs look like nails.

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#78
post #65

Earlier quoted context omitted.

it also operates at (less than) the speed of sound.

The brain is slower than the speed of sound in what medium?

The one people always mean unless they specify otherwise: Air.

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#79

The news article mentions that the student on the paper has passed away. He died of suicide in his lab just a couple weeks after the paper was submitted to Nature [0]. Grad school is hard for a lot of people. Find someone to talk to if you're struggling. [0] https://padailypost.com/2019/02/16/engineering-student-found...

I've had an idea for a while, coming from a developing continent, that it would be good to have an alternative for students where they can start working when they are doing an advanced degree, but also simply a place for high school students to acclimate to some of the aspects of the work place.

I know it sounds tangential, but this does resonate with my own roundabout way of going from studies -> work and the gross lack of fundamental science companies in Africa as a whole. I did not have depression, but rather my own cocktail of problems as a student, and a lot of it (at least for me) stemmed from clear dissonance between your current situation and an inability to anticipate a future self that is acceptable to your current self.

With depression, the situation is similar and more importantly, you become more and more subjective and people outside your situation are needed, as probably is medication, to bring you back to personal assonance with what you want in life.

The article doesn't say what the reasons were, so Wang may have had different reasons.

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#80
post #79

The news article mentions that the student on the paper has passed away. He died of suicide in his lab just a couple weeks after the paper was submitted to Nature [0]. Grad school is hard for a lot of people. Find someone to talk to if you're struggling. [0] https://padailypost.com/2019/02/16/engineering-student-found...

I've had an idea for a while, coming from a developing continent, that it would be good to have an alternative for students where they can start working when they are doing an advanced degree, but also simply a place for high school students to acclimate to some of the aspects of the work place. I know it sounds tangential, but this does resonate with my own roundabout way of going from studies -> work and the gross…

With depression, the "why" is never easy to suss out, even when something is stated, is that really the reason?
Post reply on HN