Live data from Hacker News

Machine translation of cortical activity to text with encoder–decoder framework

nature.com

21–30 of 93 posts

Re: Machine translation of cortical activity to text with encoder–decoder framework

#21
post #17

Earlier quoted context omitted.

What's your opinion on Neuralink?

Neuralink is even more speculative - they want to embed like a thousand tiny metal filaments deep into your brain.

How do you place electrodes without metal filaments? It’s been validated in monkeys.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#22
post #9

This is cool. For those who are not super familiar with language processing, I think it's good to point out the limitations of what's been done here though. They mention that professional speech transcription has word error rate around 5%, and that their method gets a WER of 3%. Sure, but the big distinction is that speech transcription must operate on an infinite number of sentences, even sentences that have never b…

Is the following quote at odds with what you are saying about 50-way classification?

"On the other hand, the network is not merely classifying sentences, since performance is improved by augmenting the training set even with sentences not contained in the testing set (Fig. 3a,b). This result is critical: it implies that the network has learned to identify words, not just sentences, from ECoG data, and therefore that generalization to decoding of novel sentences is possible."

Re: Machine translation of cortical activity to text with encoder–decoder framework

#23
post #19

Really cool to see progress made here but this won't be available for public use any time soon (likely decades). One of the biggest challenges with decoding brain signals is getting a large number sensors that detect voltages from a very localized region of the brain. This study was done with ECoG (Eletro-Cortico-Gram) which involves implanting small electrodes directly on the surface of the brain. Nearly all consume…

No, actually high density electrode implants are right around the corner. Watch the neuralink press event.

The limiting factor here isn't implant technology. The limiting factor is the rate we can implant electrodes and do new science.

Don't get me wrong, I love Elon as much as the next fanboy but there are serious ethical implications at play here that you can't just engineer your way around.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#24
post #12
post #7

Earlier quoted context omitted.

They use 250 ECG electrodes as input. I think that means it's above the skin, so not invasive.

You are mistaken. They use ECoG which is intracranial electroencephalography. These are electrodes placed on exposed surface of the brain. https://en.wikipedia.org/wiki/Electrocorticography

Thanks for the info!

Re: Machine translation of cortical activity to text with encoder–decoder framework

#25
post #21
post #17

Earlier quoted context omitted.

Neuralink is even more speculative - they want to embed like a thousand tiny metal filaments deep into your brain.

How do you place electrodes without metal filaments? It’s been validated in monkeys.

Is that true?

There was a throwaway comment about that at a press conference and lots of rumors (positive and negative), but the white paper only mentions “monkey” once—-and in a reference to another group’s paper.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#26

Really cool to see progress made here but this won't be available for public use any time soon (likely decades). One of the biggest challenges with decoding brain signals is getting a large number sensors that detect voltages from a very localized region of the brain. This study was done with ECoG (Eletro-Cortico-Gram) which involves implanting small electrodes directly on the surface of the brain. Nearly all consume…

What's your opinion on Neuralink?

I think they're a cool company and I even thought about applying. However the limiting factor here isn't the implant technology. Lots of groups have been building really great stuff for years (groups around the world have been recording from neurons in living animals for decades). The issue is there are ethical implications for performing such an invasive brain surgery on a healthy person. Right now the limit factor is finding suitable candidates.

The ECoG implants are usually done to pinpoint where seizers start in the brain. The surgeons already have a good idea of the rough area from EEG but to zero in on the exact locus they need ECoG.

So to find a candidate for a study you need someone who has epilepsy, their epilepsy needs to be bad enough to merit brain surgery, and the epileptic center cannot be the brain region you care about but it needs to be close enough to your target region that the same ECoG will cover both brain areas.

So again I'm all for pushing this science forward. The more we learn about how the brain works the more we'll understand what makes us human. However, this isn't a technology problem right now. Its an ethical and medical one.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#27
post #17

Earlier quoted context omitted.

What's your opinion on Neuralink?

Neuralink is even more speculative - they want to embed like a thousand tiny metal filaments deep into your brain.

This isn't new at all. Groups have been doing this for decades. Neuralink wants to mass produce the devices.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#28
post #15

Really cool to see progress made here but this won't be available for public use any time soon (likely decades). One of the biggest challenges with decoding brain signals is getting a large number sensors that detect voltages from a very localized region of the brain. This study was done with ECoG (Eletro-Cortico-Gram) which involves implanting small electrodes directly on the surface of the brain. Nearly all consume…

Stupid question but could you do something very simple like on/off-switch with EEG and something similar than what they have done here?

Not sure how that would change anything. Think of the brain as a crowd at a soccer field and to decode the brain you'd need a mic on each person to capture every conversation. ECoG is like having a mic shared between every 100 people. While EEG is like having a single mic that is 10K feet over the soccer stadium.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#29
post #15

Really cool to see progress made here but this won't be available for public use any time soon (likely decades). One of the biggest challenges with decoding brain signals is getting a large number sensors that detect voltages from a very localized region of the brain. This study was done with ECoG (Eletro-Cortico-Gram) which involves implanting small electrodes directly on the surface of the brain. Nearly all consume…

Stupid question but could you do something very simple like on/off-switch with EEG and something similar than what they have done here?

Yes, see EPOC

Re: Machine translation of cortical activity to text with encoder–decoder framework

#30
post #21
post #17

Earlier quoted context omitted.

Neuralink is even more speculative - they want to embed like a thousand tiny metal filaments deep into your brain.

How do you place electrodes without metal filaments? It’s been validated in monkeys.

you need something that conducts electricity. There are groups that use silicon implants.
Post reply on HN