Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine translation of cortical activity to text with encoder–decoder framework

nature.com

Post reply on HN
Print view View original

51–60 of 93 posts

Re: Machine translation of cortical activity to text with encoder–decoder framework

#51

Post by BickNowstrom »

Ajedi32 wrote: post #37
bognition wrote: post #26

Earlier quoted context omitted.

I think they're a cool company and I even thought about applying. However the limiting factor here isn't the implant technology. Lots of groups have been building really great stuff for years (groups around the world have been recording from neurons in living animals for decades). The issue is there are ethical implications for performing such an invasive brain surgery on a healthy person. Right now the limit factor…

> However, the limiting factor here isn't the implant technology. [...] The issue is there are ethical implications for performing such an invasive brain surgery on a healthy person. That sounds like a limitation of the implant technology to me. The reason there are ethical problems with performing invasive brain surgery on a healthy person is because the risks and downsides of currently available implant technology…

There are quite a few ethical problems. On the reading side, there is interrogation (of suspected spies or terrorists), which is already decades underway with fMRI, and brainjacking (a keylogger for your thoughts). As for deep brain stimulation, this is researched to create fearless soldiers, change sexuality, extremely painful torture, radio control movements, and alter emotional states.

Even for patients who suffer from severe health conditions such as OCD, depression, or tremors, the treatment can be very disruptive and emotional well-being may decrease, even with successful treatment of the condition (the doctor is happy, the patient less so).

The dual use possibilities of this technology are extremely scary, and the involvement of Facebook (supposedly for VR) and DARPA (supposedly to treat anxiety and PTSD in soldiers) does not bode too well.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#52

Post by hrgiger »

linschn wrote: post #50
hrgiger wrote: post #44

I have tried something similar 5 years ago using the meditation device from choosemuse.com. It was the cheapest option and provided hackable interface that you had access all the data. Then I wrote a small mobile app connects to headset. Application was picking and showing a single random word from "hello world my name is hrgiger" then showing a greeen light, when I see green light, i think about the word and blink,…

40% over 6 classes is way above a random baseline. This is actually pretty cool. Congratulations!

Oh thank you! I wasnt even aware those numbers were promising.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#53

Post by mattkrause »

2008guy wrote: post #38
mattkrause wrote: post #25

Earlier quoted context omitted.

Is that true? There was a throwaway comment about that at a press conference and lots of rumors (positive and negative), but the white paper only mentions “monkey” once—-and in a reference to another group’s paper.

Elon musk said on stage that they had put it in a monkey and had the monkey move a cursor with it. I’ve been following musk very closely since 2010 and he’s never blatantly lied so I think this will not be the first time. He has made errors about timelines and etc but never, ever blatantly lied about something so concrete. And it’s plausible to boot.

Not to swing my credentials around, but I actually record neural activity, from monkey brains, for a living.

It can be unexpectedly hard in so many different ways. The dura covering the monkey brain is much tougher, the brain itself is larger, more convoluted, and moves more, even just from breathing and heartbeats). The animals have busy, clever little fingers, so the interface itself needs to be mechanically robust and durable because these implants need to last for years.

I certainly want this to be true: with the exception of neuropixels, electrode technology has been depressingly stagnant. On the other hand, I need to see data before I get too excited and if I did have it, I'd be shouting it from the rooftops.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#54

Post by turingbike »

bognition wrote: post #10

Really cool to see progress made here but this won't be available for public use any time soon (likely decades). One of the biggest challenges with decoding brain signals is getting a large number sensors that detect voltages from a very localized region of the brain. This study was done with ECoG (Eletro-Cortico-Gram) which involves implanting small electrodes directly on the surface of the brain. Nearly all consume…

There was a great article on the front page yesterday, When to Assume Neural Networks Can Solve a Problem, https://news.ycombinator.com/item?id=22717367 . Case 2 is very relevant here, basically: if you have solved the problem with access to lots of data, usually you can adapt to a lower-data regime.

I am way out of my element talking about brain surgery and sensors. However, one thing that I do well is say "you shouldn't bet against neural networks", which is a great way to be right on a few-year time horizon.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#55

Post by carapace »

hrgiger wrote: post #49
andai wrote: post #46

Earlier quoted context omitted.

The Muse headband looks like it covers a pretty small area of the head right? Other products in a similar price range cover more or less the whole scalp.

Correct, if I recall correctly it has 8 sensors, 4 forehead and 4 ear side. Previous year I wanted to try again but seemed like they drop linux support and asked to reach them via mail to have sdk, so I didnt bother. If i would go now I would go with openbci project I guess

I've got a muse on my desk right now, it works-for-me on Ubuntu 16.04 with a BLE dongle and the BGAPIbackend of pygatt.

FWIW, the main advantage of the muse design is that by going across the forehead it avoids interference from your hair, if I understand correctly.

The NeuroTechX ppl are working to get some demo notebooks working with muse. https://github.com/NeuroTechX/eeg-notebooks https://neurotechx.com/

Re: Machine translation of cortical activity to text with encoder–decoder framework

#56

Post by Balgair »

2008guy wrote: post #40
empath75 wrote: post #33

Earlier quoted context omitted.

> One engineering challenge is that the brain's chemical environment can cause many plastics to gradually deteriorate. Another challenge to chronic electrode implants is the inflammatory reaction to brain implants. Transmission of chemical messengers via neurons is impeded by a barrier-forming glial scar that occurs within weeks after insertion followed by progressive neurodegeneration, attenuating signal sensitivity…

The inflammation and damage is seen in traditional arrays that are large and rigid. A thread with low enough moment of inertia will likely not cause as much damage. And the damage is only important if you put the electrodes in an important place... we currently screw two giant lag screws into people’s heads and call it “deep brain stimulation” so I feel optimistic about the long game. But you’d still be right to be w…

> A thread with low enough moment of inertia will likely not cause as much damage.

Shear forces cause glial scarring?

Re: Machine translation of cortical activity to text with encoder–decoder framework

#57

Post by stainforth »

2008guy wrote: post #40
empath75 wrote: post #33

Earlier quoted context omitted.

> One engineering challenge is that the brain's chemical environment can cause many plastics to gradually deteriorate. Another challenge to chronic electrode implants is the inflammatory reaction to brain implants. Transmission of chemical messengers via neurons is impeded by a barrier-forming glial scar that occurs within weeks after insertion followed by progressive neurodegeneration, attenuating signal sensitivity…

The inflammation and damage is seen in traditional arrays that are large and rigid. A thread with low enough moment of inertia will likely not cause as much damage. And the damage is only important if you put the electrodes in an important place... we currently screw two giant lag screws into people’s heads and call it “deep brain stimulation” so I feel optimistic about the long game. But you’d still be right to be w…

Can you continue about your opposition to it existing?

Re: Machine translation of cortical activity to text with encoder–decoder framework

#58

Post by hyyggnj »

leggomylibro wrote: post #20

It looks cool, but they trained their models on people reading printed sentences out loud. Would that actually translate to decoding the process of turning abstract thoughts into words? The researchers also note that their models are vulnerable to over-fitting because of the paucity of training data, and they only used a 250-word vocabulary. Neuralink also has a strong commercial incentive to inflate the results, so…

The speaking aloud is very suspicious. Why do subjects need to speak aloud? Are they actually decoding neural signals or just picking up artifacts introduced by the physical act of speaking (i.e. electrodes vibrating due to sound, etc)?

Re: Machine translation of cortical activity to text with encoder–decoder framework

#59

Post by carapace »

I'm pretty sure you can get that with a HD camera or two and some hypnosis plus off-the-shelf ML.

One of the very first things I learned when I was studying hypnosis was to induce a simple binary signal from the unconscious. (Technically it's trinary: {y,n,mu} https://en.wikipedia.org/wiki/Mu_(negative) )

(In my case my right arm would twitch for "yes", left for "no", no twitch for "mu" (I don't want to go on a long tangent about all the various shades of meaning there, suffice it to say it's a form of "does not compute."))

Anyway, it would be trivial to set up one or more binary signals, and detect them via switches or, these days, HD cameras and ML. You could train your computer to "read" your mind from very small muscular contractions/relaxations of your face. (The primary output channel, even before voice, of the brain, eh?)

Or you could just set up a nine-bit parallel port (1 byte + clock) and hypnotize yourself to emit ASCII or UTF_8 directly. That would be much much simpler because it's so much easier and faster to write mind software than computer software (once you know how.) And you could plug yourself into any USB port and come up as a HID (mouse & keyboard.)

I'll say it again: when you connect a brain to a computer the more sophisticated information processing unit is the point of greatest leverage. Trying to get the computer to do the work is like attaching horses to the front of your truck to tow it. Put the horses in the back and let the engine tow them.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#60

Post by briga »

It seems like this field is at about the same stage of progress as image recognition was in the 90s when researchers were trying to getting a handle on MNIST-type tasks.

I wonder how much the language embeddings learned by the transformer are reflected in the actual physical structure of the brain? Could it be that the transformer is making the same sort of representations as those in the brain, or is it learning entirely new representations? My guess is that it's doing something quite different from what the brain is doing, although I wouldn't rule out some sort of convergence. Either way, this is a fascinating branch of research both for AI and the cognitive sciences.

Post reply on HN