Live data from Hacker News

Machine translation of cortical activity to text with encoder–decoder framework

nature.com

31–40 of 93 posts

Re: Machine translation of cortical activity to text with encoder–decoder framework

#31

Fascinating work but far from what some might hope from reading only the title. The translation is restricted to a vocabulary of 30 to 50 unique sentences.

More than enough to control window focus on my computer and such. I'd be happy to have a system that responded to a few hundred thoughts: "Left desktop", "Right desktop", "Last focused window", "Lock screen", "What time is it?"

Re: Machine translation of cortical activity to text with encoder–decoder framework

#32

Really cool to see progress made here but this won't be available for public use any time soon (likely decades). One of the biggest challenges with decoding brain signals is getting a large number sensors that detect voltages from a very localized region of the brain. This study was done with ECoG (Eletro-Cortico-Gram) which involves implanting small electrodes directly on the surface of the brain. Nearly all consume…

Has there been any work to use a neural network to generate/simulate ECoG signal output from an EEG signal input? (My Google-fu only gives definitions and distinctions for ECoG and EEG). Almost sounds similar in concept to deep learning super sampling (DLSS), i.e., taking a low resolution image/signal (EEG) and using a neural network to generate/simulate a high resolution image/signal (ECoG).

Re: Machine translation of cortical activity to text with encoder–decoder framework

#33
post #19

Really cool to see progress made here but this won't be available for public use any time soon (likely decades). One of the biggest challenges with decoding brain signals is getting a large number sensors that detect voltages from a very localized region of the brain. This study was done with ECoG (Eletro-Cortico-Gram) which involves implanting small electrodes directly on the surface of the brain. Nearly all consume…

No, actually high density electrode implants are right around the corner. Watch the neuralink press event.

> One engineering challenge is that the brain's chemical environment can cause many plastics to gradually deteriorate. Another challenge to chronic electrode implants is the inflammatory reaction to brain implants. Transmission of chemical messengers via neurons is impeded by a barrier-forming glial scar that occurs within weeks after insertion followed by progressive neurodegeneration, attenuating signal sensitivity. Furthermore, the thin electrodes which Neuralink uses, are more likely to break than thicker electrodes, and currently cannot be removed when broken or when rendered useless after glial scar forming.

Yeah, you're not putting that in my brain.

https://en.wikipedia.org/wiki/Neuralink

Re: Machine translation of cortical activity to text with encoder–decoder framework

#34

Really cool to see progress made here but this won't be available for public use any time soon (likely decades). One of the biggest challenges with decoding brain signals is getting a large number sensors that detect voltages from a very localized region of the brain. This study was done with ECoG (Eletro-Cortico-Gram) which involves implanting small electrodes directly on the surface of the brain. Nearly all consume…

Seismologist here. The problems you describe sound reminiscent of those in seismic imaging. Ideally you'd bury geophones below the weathered layer (typically 3 m) at the surface, to get ideal coupling. In practice that's not economic at scale, and so you plant them on the surface, and account for the non-linear wave transmission through the weathered layer by clever math and by collecting more samples.

There's been a forty-year evolution in these techniques. The cheap, noisy technique might prevail if scientists keep refining the craft by tiny improvements.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#35

Really cool to see progress made here but this won't be available for public use any time soon (likely decades). One of the biggest challenges with decoding brain signals is getting a large number sensors that detect voltages from a very localized region of the brain. This study was done with ECoG (Eletro-Cortico-Gram) which involves implanting small electrodes directly on the surface of the brain. Nearly all consume…

Has there been any work to use a neural network to generate/simulate ECoG signal output from an EEG signal input? (My Google-fu only gives definitions and distinctions for ECoG and EEG). Almost sounds similar in concept to deep learning super sampling (DLSS), i.e., taking a low resolution image/signal (EEG) and using a neural network to generate/simulate a high resolution image/signal (ECoG).

The problem sounds similar but it's completely different

DLSS kinda works because we "know" what is what thing in a photo.

EEG to ECOG would be trying to figure out a painting (that could be anything, from any painter) from a significant distance behind a frosted glass

Re: Machine translation of cortical activity to text with encoder–decoder framework

#36
post #19

Really cool to see progress made here but this won't be available for public use any time soon (likely decades). One of the biggest challenges with decoding brain signals is getting a large number sensors that detect voltages from a very localized region of the brain. This study was done with ECoG (Eletro-Cortico-Gram) which involves implanting small electrodes directly on the surface of the brain. Nearly all consume…

No, actually high density electrode implants are right around the corner. Watch the neuralink press event.

What Neuralink has done so far is pretty impressive, but I think "decades" is still a pretty reasonable estimate for when that technology will be available for use cases that aren't medically necessary.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#37

Earlier quoted context omitted.

What's your opinion on Neuralink?

I think they're a cool company and I even thought about applying. However the limiting factor here isn't the implant technology. Lots of groups have been building really great stuff for years (groups around the world have been recording from neurons in living animals for decades). The issue is there are ethical implications for performing such an invasive brain surgery on a healthy person. Right now the limit factor…

> However, the limiting factor here isn't the implant technology. [...] The issue is there are ethical implications for performing such an invasive brain surgery on a healthy person.

That sounds like a limitation of the implant technology to me. The reason there are ethical problems with performing invasive brain surgery on a healthy person is because the risks and downsides of currently available implant technology are significant. If getting a brain implant were as cheap, easy, and safe as getting a tattoo, the ethical problem would be largely solved.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#38
post #21

Earlier quoted context omitted.

How do you place electrodes without metal filaments? It’s been validated in monkeys.

Is that true? There was a throwaway comment about that at a press conference and lots of rumors (positive and negative), but the white paper only mentions “monkey” once—-and in a reference to another group’s paper.

Elon musk said on stage that they had put it in a monkey and had the monkey move a cursor with it. I’ve been following musk very closely since 2010 and he’s never blatantly lied so I think this will not be the first time. He has made errors about timelines and etc but never, ever blatantly lied about something so concrete. And it’s plausible to boot.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#40
post #19

Earlier quoted context omitted.

No, actually high density electrode implants are right around the corner. Watch the neuralink press event.

> One engineering challenge is that the brain's chemical environment can cause many plastics to gradually deteriorate. Another challenge to chronic electrode implants is the inflammatory reaction to brain implants. Transmission of chemical messengers via neurons is impeded by a barrier-forming glial scar that occurs within weeks after insertion followed by progressive neurodegeneration, attenuating signal sensitivity…

The inflammation and damage is seen in traditional arrays that are large and rigid. A thread with low enough moment of inertia will likely not cause as much damage. And the damage is only important if you put the electrodes in an important place... we currently screw two giant lag screws into people’s heads and call it “deep brain stimulation” so I feel optimistic about the long game. But you’d still be right to be wary. And if I had it my way, this technology would never be allowed to exist at all...
Post reply on HN