Live data from Hacker News

Memristor Breakthrough: First Single Device to Act Like a Neuron

spectrum.ieee.org

61–70 of 112 posts

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#61

Off topic. When something is on the tip of my tongue, I don't search for it because I have a strange belief that what my mind is trying to do to remember it is fire off neurons associated with the thought or idea, near it, to create the action potential. It is strange that I will eventually remember it within hours or the next day. When the action potential fires the neuron, dendrites are formed between neurons stren…

I feel like if this were true (searching doesn’t strengthen the original memory), it would lead to a dramatic difference in a cohort’s ability to remember trivial information that they learned a long time ago, but it also seems really hard to perform a trial for something like this.

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#62
post #22

Earlier quoted context omitted.

That just means an FET isn't the right device for analog circuitry. Maybe this memristor device will be more efficient.

It's a bit more fundamental than that. A FET goes between say, 0.01 Ohms and 1,000,000 Ohms for fully on and fully off. The times in between are where it passes the most power through it and that's because of how Ohms law and Power being current times voltage (ignoring differing phase between current and voltage in AC circuits). This all still remains true even for the memristors. You can do analog stuff with higher…

It's my understanding that, to some extent, nerves and muscles operate with hysteresis based on charge thresholds, and rely on pulse-density encoding for variability rather than proportional voltage or current modulation. Presumably, that seems to be why your muscles shake when under heavy strain.

It sorta makes sense, considering your body is generally electrically conductive everywhere. Signals propagate as waves of charge sustained by cells opening and closing little pores that selectively release ions. "Conductors" have pores, and "insulators" don't. That's also why nerve signals are relatively slow to propagate vs. straight up electricity. That's also why you can measure muscle activity via an ECG; you're measuring the transient voltages being generated by the waves of charge moving around. The voltages themselves don't have much meaning to your body. Of course, if you apply enough voltage to build up a charge inside a muscle, you can cause it to actuate.

I recently had to get a pacemaker, and apparently the particular lower threshold to trigger my heart muscle to contract is around 0.7V for 350uS, at whatever impedance the lead happens to be. Below that, and my heart muscle does nothing. Anything above that, any my heart muscle does a full beat. The device applies a 2V pulse so that there's plenty of margin, and can go up to 5V if needed, in case the lead's impedance increases. The lower the voltage the better, in terms of battery life. The cool thing is that the device can safely coexist with any natural electrical activity my faulty nerves may have, since the muscle simply responds to whichever pulse happens to arrive first.

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#63
post #7

For neural networks, it makes sense to skip pure digital design. When I learned how to design an ALU to say, add, and wait for the propagation of carry bits that’s like O(n) where n is the number of bits in the number, it made me want to just use superposition for addition, which is physical and instantaneous. Of course, that has all sorts of other problems that make it worse (so much worse). Once you learn how to sl…

A bottom-up rewrite of modern computer engineering for performance and capability would probably qualify as one of our largest engineering endeavors in human history. But I would totally be on board for that. There would inevitably be some really interesting discoveries from it, no doubt.

Here is an attempt https://dspace.mit.edu/bitstream/handle/1721.1/78468/8340892...

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#64

Unfortunately, we do not know how neurons learn, compute and represent information in biological neural networks. Though I feel that there would be great potential in figuring this out - the energy efficiency and capabilities of the brain are truly amazing.

Huh?

>Learn

Depends on the neuron, but things like NMDA and AMPA receptors are very well studied at this point and are the primary portion of synaptic plasticity, again, depending on the neuron: https://en.wikipedia.org/wiki/AMPA_receptor

>Compute

Again, depends on the neuron, but dendritic integration is a classic example: https://www.sciencedirect.com/topics/neuroscience/dendritic-...

> Represent information

The do so in the frequency space. But again, it's highly dependent on the system of neurons : https://en.wikipedia.org/wiki/Neural_coding

That said, holy cow yes. The brain is really something when it comes to power usage.

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#65

Earlier quoted context omitted.

The brain is still way more efficient and parallel then a GPU (e.g. 12W compared to 280W), so being able to duplicate it closer sounds like a pretty compelling advancement.

it also operates at (less than) the speed of sound.

The brain is slower than the speed of sound in what medium?

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#66
post #30

I'm not convinced that spiking behavior is all that's needed to really emulate what's going on in our neurons. Modern hardware designs are not really amenable to replicating the number or structure of synapses that we have, muchless the extremely complicated biochemistry that controls how and when they fire. It's good to see progress being made, but we really need to tone down the hyperbole in the popular scientific…

No, they mean that the memristor acts like the synapse.

It was the last remaining basic two wire circuit element (the others being the resistor, capacitor, and inductor). We could model cells, up to the memristive portion, entirely with basic circuit components (albeit in very complicated arrangements). We were missing that part of our bits boxes. Now we aren't, well, sorta. It'll be a while before we get these in our hands.

Also, these things are a bit bigger than just modern hardware. It's as if you just added a new primary color to Bob Ross; everything changes. We're going to need to redesign computers from the electrical-engineering-ground up.

Well maybe, it'll be a few years (hopefully not decades)

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#67
post #57
post #25

"Acts like a neuron" is BS. Neurons are more like individual computers themselves, with all sorts of activity at the subcellular level.

Fun fact: there are about a hundred trillion atoms in every neuron.

At first I was like: "There's no freakin' way"

But then, sure enough wolfram alpha: 100,000,000,000,000 carbon atoms is ~0.167 nano mols. Or ~2 nanograms of carbon. Dang.

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#68
post #51
post #48

Earlier quoted context omitted.

> the power efficiency of analogue circuitry is dramatically worse The human brain would like to have a word with you.

At the slow "clock speed" the brain runs at, it produces a lot of heat. A quick search turns up 10-20 watts as a rough estimate. It's still petty competitive given what it's capable of.

Why do you say slow "clock speed"? Can we even use clock speed w.r.t brain? Genuinely curious.

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#69

12 years ago, on HN: "The memristor—the functional equivalent of a synapse—could revolutionize circuit design" https://news.ycombinator.com/item?id=397335

It's fascinating that none of the comments mention neural network applications. Seems like these people came a bit too early to the party.

We've left biomimicry in the dust a fair time ago - neurons do not work like relus after all (nor do they work like memristors), but relus are great at solving problems. None of our current NN applications would even want something that behaves like a neuron. You'd essentially have to start from scratch. It's not even clear if behaves-like-a-neuron is better or not to be worth the effort. (And that's disregarding the entire fact that memristors do not behave like neurons either!)

Re: Memristor Breakthrough: First Single Device to Act Like a Neuron

#70
post #47
post #32

If you find water analogies helpful for circuit elements, here is one for the memristor: "The closest analogy I can think of is a sand filter, an item of apparatus used in water-purification plants. As contaminated water flows through a bed of sand and gravel, sediment gradually clogs the pores of the filter and thereby increases resistance. Reversing the flow flushes out the sediment and reduces resistance." -- http…

Isn’t that like a capacitor? As it charges the voltage increases and the current drops until it reaches zero. Then if you discharge it you can start over again.

Capacitors bridge voltage and charge, yes. In exactly the same way, memristors bridge charge and flux.
Post reply on HN