Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using a BCI with LLM for enabling ALS patients to speak again with family

thevccorner.substack.com

Post reply on HN
Print view View original

11–20 of 24 posts

Re: Using a BCI with LLM for enabling ALS patients to speak again with family

#11

Post by Sanzig »

y-curious wrote: post #9
xk_id wrote: post #8

Those non-invasive headbands (which work very differently from implanted electrodes) are notoriously inaccurate at recording brain signals. Even scientific studies, which use advanced setups like the 10-20 system for scalp EEG, face unsolved challenges in removing noise from the data and in using the data to reconstruct underlying brain activity [0] – let alone making meaningful inferences about it. Patients with loc…

I've spoken to a lot of smart people on the topic of EEG (I'm in a very related field). I agree with you. It's an extremely powerful tool for diagnosis of a limited range of conditions but it is not magic. Electrical signal gets attenuated heavily when signals are not on the outside of the brain. Even still, a headband like this is susceptible to noise from movement and other factors. You either need to correct for t…

What they need to provide is surveys from the patients without the device (even locked in patients can often communicate slowly via eye-scan interfaces). How well do the patients rate the system at aligning with what they want to say?

If they don't find that it aligns at all, then honestly that is worse than nothing. Imagine being locked in and your family communicating with an LLM pretending to be you - all while you have to watch and can't do anything about it.

Re: Using a BCI with LLM for enabling ALS patients to speak again with family

#12

Post by Sanzig »

throwup238 wrote: post #10
Sanzig wrote: post #7

Earlier quoted context omitted.

Agreed - I don't want to come across as negative, but this is certainly in the "extraordinary claims" category for me right now. If this works, it's obviously huge , but I really want to see third party validation before I can let go of my skepticism. I would be very curious to hear about interviews with the patients (conducted through their current means of communication, eg: eye gaze interfaces). Are they finding t…

Even if the LLM hallucinates every word, just knowing when to say something versus stay quiet based on EEG data would be a huge breakthrough.

If that's all they were doing - showing when the patient wanted to speak - that would be fine. Presenting speech as attributable to that patient, though? That feels irresponsible without solid evidence, or at least informing the families of the risk that the interface may be just totally hallucinating. Imagine someone talking to an LLM they think is their loved one, all while that person has to watch.

Re: Using a BCI with LLM for enabling ALS patients to speak again with family

#13

Post by anonzzzies »

Sanzig wrote: post #11
y-curious wrote: post #9

Earlier quoted context omitted.

I've spoken to a lot of smart people on the topic of EEG (I'm in a very related field). I agree with you. It's an extremely powerful tool for diagnosis of a limited range of conditions but it is not magic. Electrical signal gets attenuated heavily when signals are not on the outside of the brain. Even still, a headband like this is susceptible to noise from movement and other factors. You either need to correct for t…

What they need to provide is surveys from the patients without the device (even locked in patients can often communicate slowly via eye-scan interfaces). How well do the patients rate the system at aligning with what they want to say? If they don't find that it aligns at all, then honestly that is worse than nothing. Imagine being locked in and your family communicating with an LLM pretending to be you - all while yo…

It might be beneficial to the family though, but indeed not to you.

Re: Using a BCI with LLM for enabling ALS patients to speak again with family

#14

Post by throwup238 »

Sanzig wrote: post #12
throwup238 wrote: post #10

Earlier quoted context omitted.

Even if the LLM hallucinates every word, just knowing when to say something versus stay quiet based on EEG data would be a huge breakthrough.

If that's all they were doing - showing when the patient wanted to speak - that would be fine. Presenting speech as attributable to that patient, though? That feels irresponsible without solid evidence, or at least informing the families of the risk that the interface may be just totally hallucinating. Imagine someone talking to an LLM they think is their loved one, all while that person has to watch.

You’ll get no argument from me there. The whole LLM part seems like a gimmick unless it’s doing error correction on a messy data stream like a drunk person fat fingering in a question to ChatGPT except with an EEG. It might be a really fancy autocorrect.

I’m just saying that EEG data is so unreliable and requires so much calibration/training per person that reliably isolating speech in paralyzed patient would be a significant development.

Re: Using a BCI with LLM for enabling ALS patients to speak again with family

#16

Post by sakras »

xk_id wrote: post #8

Those non-invasive headbands (which work very differently from implanted electrodes) are notoriously inaccurate at recording brain signals. Even scientific studies, which use advanced setups like the 10-20 system for scalp EEG, face unsolved challenges in removing noise from the data and in using the data to reconstruct underlying brain activity [0] – let alone making meaningful inferences about it. Patients with loc…

I spent a long time trying to do some sort of machine learning over the OpenBCI helmet's data with the eventual goal of moving a cursor, but I didn't seem to get anywhere. The data was indeed _extremely_ noisy, and I don't believe my model ever converged to anything useful.

That said, I was just a high schooler and so my method of collecting training data was to run the script and "think really hard about moving left". Probably could have been a good deal more sophisticated too.

Re: Using a BCI with LLM for enabling ALS patients to speak again with family

#17

Post by JPLeRouzic »

It's good that people are trying to help ALS patients with technology, but most ALS patients are not in the terminal phase of their disease.

Most pALS have difficulties speaking, their voice is weak and sometimes there is a pronunciation problem. Many pALS use simple assistive devices like a personal voice amplifier, or a text to speech device.

Sometimes the problem is that "normal" people think pALS are deaf so they speak loudly but the patient can hear quite normally. Or doctors or other people ignore the patient and insist on speaking to the carer as if the patient cannot comprehend it.

https://alsnewstoday.com/forums/forums/topic/artificial-voic...

https://alsnewstoday.com/columns/microphone-voice-hear/

Re: Using a BCI with LLM for enabling ALS patients to speak again with family

#18

Post by frereubu »

This sounds like hocus pocus to me. I presume the "headset" band is some kind of EEG given that it's "non-invasive", and if so EEG signals are very crude, far too crude to map to anything like language beyond, perhaps, "I'm sad" or "I'm excited". They also only measure brain activity at the surface of the brain, and have no information whatsoever on anything below the first centimetre or so, if that. EEG headsets that aren't a full skullcap with sensors that have some kind of conductive gel applied to each one and a tangle of wires coming off the back are pretty much entirely useless, particularly when they're claiming signals as granular as language. Not only that, the "or" in the phrase "sensors embedded in a wristband or headband capture input from your brain" suggest that they think a wristband alone is enough to "capture input from your brain", which is an outrageous claim. Just because they hook this up to an LLM that churns out language doesn't mean much - we all know that LLMs are far too happy to generate language for its own sake. Unless they show a great deal more detail about how this is supposed to work this should be regarded as AI-induced magical thinking, and they're making wildly irresponsible claims that could generate a lot of false hope.

Re: Using a BCI with LLM for enabling ALS patients to speak again with family

#19

Post by NitpickLawyer »

sakras wrote: post #16
xk_id wrote: post #8

Those non-invasive headbands (which work very differently from implanted electrodes) are notoriously inaccurate at recording brain signals. Even scientific studies, which use advanced setups like the 10-20 system for scalp EEG, face unsolved challenges in removing noise from the data and in using the data to reconstruct underlying brain activity [0] – let alone making meaningful inferences about it. Patients with loc…

I spent a long time trying to do some sort of machine learning over the OpenBCI helmet's data with the eventual goal of moving a cursor, but I didn't seem to get anywhere. The data was indeed _extremely_ noisy, and I don't believe my model ever converged to anything useful. That said, I was just a high schooler and so my method of collecting training data was to run the script and "think really hard about moving left…

If it's any consolation for young you, this is a really hard problem, even with electrodes implanted in the brain. There's an amazing podcast by lex with the team from NeuralLink, and they go into depth on how even with good neuron signals, there's still a lot of work on the software side to "moving a cursor". The first recipient of their implant still has to do a 10-30min calibration run every morning to be able to move a cursor reliably on the screen. So all in all, don't beat yourself up, it's a really hard problem even with good data.

Re: Using a BCI with LLM for enabling ALS patients to speak again with family

#20

Post by OJFord »

throwup238 wrote: post #14
Sanzig wrote: post #12

Earlier quoted context omitted.

If that's all they were doing - showing when the patient wanted to speak - that would be fine. Presenting speech as attributable to that patient, though? That feels irresponsible without solid evidence, or at least informing the families of the risk that the interface may be just totally hallucinating. Imagine someone talking to an LLM they think is their loved one, all while that person has to watch.

You’ll get no argument from me there. The whole LLM part seems like a gimmick unless it’s doing error correction on a messy data stream like a drunk person fat fingering in a question to ChatGPT except with an EEG. It might be a really fancy autocorrect. I’m just saying that EEG data is so unreliable and requires so much calibration/training per person that reliably isolating speech in paralyzed patient would be a si…

Definitely, seems like the wrong tool even, or not the right first one, surely you need some sort of big classifier for EEG patterns to words or thoughts/topics; then if used an LLM it would be 'clean up this nonsense into coherent sentences keeping the spirit of the ideas or topics that are mentioned'?
Post reply on HN