Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine translation of cortical activity to text with encoder–decoder framework

nature.com

Post reply on HN
Print view View original

81–90 of 93 posts

Re: Machine translation of cortical activity to text with encoder–decoder framework

#81

Post by 2008guy »

Balgair wrote: post #56
2008guy wrote: post #40

Earlier quoted context omitted.

The inflammation and damage is seen in traditional arrays that are large and rigid. A thread with low enough moment of inertia will likely not cause as much damage. And the damage is only important if you put the electrodes in an important place... we currently screw two giant lag screws into people’s heads and call it “deep brain stimulation” so I feel optimistic about the long game. But you’d still be right to be w…

> A thread with low enough moment of inertia will likely not cause as much damage. Shear forces cause glial scarring?

Yes

Re: Machine translation of cortical activity to text with encoder–decoder framework

#82

Post by dnautics »

kensai wrote: post #71
hughw wrote: post #34

Earlier quoted context omitted.

Seismologist here. The problems you describe sound reminiscent of those in seismic imaging. Ideally you'd bury geophones below the weathered layer (typically 3 m) at the surface, to get ideal coupling. In practice that's not economic at scale, and so you plant them on the surface, and account for the non-linear wave transmission through the weathered layer by clever math and by collecting more samples. There's been a…

Neurologist here. There is an inherent problem of physics. The skull acts as a low-pass filter of the brain's activity. You lose a lot of information if you don't go under the skull. I don't think EEGs can really give the spatial and temporal resolution we really need to extract the necessary information for thought decoding (or encoding).

you don't need to decode thought, you just need to decode information at some really low bitrate to export information (say text).

Working with a NN, the brain can probably negotiate the relevant parameters (let's say frequency modulation across one channels with four bins, across 4 spots on the brain), that's 4^4 = 64 values across whatever timebox of resolution you get. That's enough to encode english letters, which itself probably is an underprovisioned mechanism of data transfer).

Re: Machine translation of cortical activity to text with encoder–decoder framework

#83

Post by dnautics »

lioeters wrote: post #41
bognition wrote: post #10

Really cool to see progress made here but this won't be available for public use any time soon (likely decades). One of the biggest challenges with decoding brain signals is getting a large number sensors that detect voltages from a very localized region of the brain. This study was done with ECoG (Eletro-Cortico-Gram) which involves implanting small electrodes directly on the surface of the brain. Nearly all consume…

I've been fascinated by this topic since a couple decades ago, when I participated in a research study involving EEG and word recognition. Progress seems to be slow but steady, with applications getting more accurate and practical. Your point about invasive implants being impractical for commercial use made me wonder.. I searched the first phrase that popped into my head, "Non-invasive Brain-Computer Interface". Look…

I don't think this is the first ever non-invasive bci. I recall decades ago there was a BCI where you could control between left and right literally by activating the appropriate hemisphere and triggering a relative potentiometer.

Extremely crude, but it worked. IIRC the instructions were "to go left, think 'go left'"; "to go right, relax". After some number of sessions (I think I remember it being "a lot") the user's brain would automatically do the correct thing.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#84

Post by dandelo1953 »

IdiocyInAction wrote: post #76
salawat wrote: post #69

Earlier quoted context omitted.

It's a social crisis waiting to happen, especially if you can end up decoding more than just what someone uses their speech centers to articulate. This is invasive to the extreme, and seems to open the door for violations of people's intimate thoughts down the road. You may not think about it much now, but if you pay any attention to things like intrusive thoughts, or even have to deal with carefully maintaining a pu…

> Before anyone points out that the same moral outrage probably occurred with the printing press; there is a big damn difference between changing someone's mind through pamphlets, and having a direct link to the limbic system to tickle on a whim. I've recently read a short story from Ted Chiang likened the development of writing to a fundamental cybernetic enhancement of the brain. I found it to be quite enlightening…

Instead of thinking of advertising as "technology" you might want to look into the military-esque research that brought it into the free market. Just like the internet, psyops was first destructed and formalized by people that value information over influence. As only one will beget the other with any statistical certainty.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#85

Post by smallnamespace »

kensai wrote: post #71
hughw wrote: post #34

Earlier quoted context omitted.

Seismologist here. The problems you describe sound reminiscent of those in seismic imaging. Ideally you'd bury geophones below the weathered layer (typically 3 m) at the surface, to get ideal coupling. In practice that's not economic at scale, and so you plant them on the surface, and account for the non-linear wave transmission through the weathered layer by clever math and by collecting more samples. There's been a…

Neurologist here. There is an inherent problem of physics. The skull acts as a low-pass filter of the brain's activity. You lose a lot of information if you don't go under the skull. I don't think EEGs can really give the spatial and temporal resolution we really need to extract the necessary information for thought decoding (or encoding).

Give the user real time feedback from the reader NN.

I wouldn’t be surprised if the biofeedback allows the user to retrain their brain to encode information in low frequency signals—the raw channel capacity necessary to transmit text is quite low, plus the reader NN can also use context if hooked up to a recurrent submodel.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#86

Post by comex »

BickNowstrom wrote: post #78
mattkrause wrote: post #72

Earlier quoted context omitted.

Got some references for "decades" of fMRI interrogation (or, really, anything else)? As far as I know, decoding is challenging with totally cooperative subjects doing simple tasks. Wiggling just a few mm is enough to completely destroy a run.

The brain can't lie (2003) > Brain scans can reveal how you think and feel, and even how you might behave. No wonder the CIA and big business are interested. https://www.theguardian.com/science/2003/nov/20/neuroscience... Technology vs. Torture (2004) > Indeed, a Pentagon agency is already funding Functional MRI research for such purposes. https://slate.com/culture/2004/08/how-technology-will-elimin... The Legality o…

None of your references say that anyone has used fMRI for interrogation, let alone for decades. At most, your first two links say that the CIA and DoD respectively are interested in using fMRI for lie detection, but that's not necessarily the same as interrogation (both agencies currently use polygraphs extensively on their own employees, a case where the subject would be cooperative), and they don't say the agencies have actually put that idea into practice. Your third link cites the first one and does suggest that using fMRI for interrogation is possible, but doesn't say whether anyone has done it. Your fourth link does not mention the government or interrogation. The other two links are not about fMRI.

So we're left with the anecdotal hearsay, which might well be true, but even if true, doesn't really show that fMRI was effective against prisoners beyond its use as a psychological trick.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#87

Post by BickNowstrom »

comex wrote: post #86
BickNowstrom wrote: post #78

Earlier quoted context omitted.

The brain can't lie (2003) > Brain scans can reveal how you think and feel, and even how you might behave. No wonder the CIA and big business are interested. https://www.theguardian.com/science/2003/nov/20/neuroscience... Technology vs. Torture (2004) > Indeed, a Pentagon agency is already funding Functional MRI research for such purposes. https://slate.com/culture/2004/08/how-technology-will-elimin... The Legality o…

None of your references say that anyone has used fMRI for interrogation, let alone for decades. At most, your first two links say that the CIA and DoD respectively are interested in using fMRI for lie detection, but that's not necessarily the same as interrogation (both agencies currently use polygraphs extensively on their own employees, a case where the subject would be cooperative), and they don't say the agencies…

I feel you are splitting hairs here. Lie detection is necessarily a part of interrogation. The references say they can do this, and have been interested in doing it, so why wouldn't they be using it? The nature of intelligence agencies demands a certain degree of secrecy/anonymity and conjecture. One author of the fourth paper, though never directly researching military or government applications, moved from the U.S. to Canada in large part due to the appropriation and funding from the military for his research.

If you allow an argument from authority of the practical use in counter terrorism interrogation, see the works of bioethicist Jonathan Marks https://scholar.google.com/citations?user=MpKuUlkAAAAJ&hl=en who in his 2007 paper cites "Correspondence between a[n anonymous] U.S. counterintelligence liaison officer and Jean Maria Arrigo" (2002-2005) https://en.wikipedia.org/wiki/Jean_Maria_Arrigo :

> Brain scan by MRI/CAT scan with contrast along with EEG tests by doctors now used to screen terrorists like I suggested a long time back. Massive brain electrical activity if key words are spoken during scans. The use of the word SEMTEX provided massive brain disturbance. Process developed by NeuroPsychologists at London’s University College and Mossad. Great results. That way we only apply intensive interrogation techniques to the ones that show reactions to key words given both in English and in their own language.

[Military interrogation takes two forms, Tactical Questioning or Detailed Interviewing. Tactical Questioning is the initial screening of detainees, Detailed Interviewing is the more advanced questioning of subjects.]

Note that I did not even make the stronger claim of decades of applied usage -- that you are making me defend and invalidate my references with -- just that it was decades underway. But above quote should satisfy even that.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#88

Post by comex »

BickNowstrom wrote: post #87
comex wrote: post #86

Earlier quoted context omitted.

None of your references say that anyone has used fMRI for interrogation, let alone for decades. At most, your first two links say that the CIA and DoD respectively are interested in using fMRI for lie detection, but that's not necessarily the same as interrogation (both agencies currently use polygraphs extensively on their own employees, a case where the subject would be cooperative), and they don't say the agencies…

I feel you are splitting hairs here. Lie detection is necessarily a part of interrogation. The references say they can do this, and have been interested in doing it, so why wouldn't they be using it? The nature of intelligence agencies demands a certain degree of secrecy/anonymity and conjecture. One author of the fourth paper, though never directly researching military or government applications, moved from the U.S.…

> The references say they can do this, and have been interested in doing it, so why wouldn't they be using it?

I'm not trying to split hairs. It is possible that, despite the interest, the technology isn't at a state where it would actually be useful in practice. However, your new citation is stronger.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#89

Post by serf »

comex wrote: post #88
BickNowstrom wrote: post #87

Earlier quoted context omitted.

I feel you are splitting hairs here. Lie detection is necessarily a part of interrogation. The references say they can do this, and have been interested in doing it, so why wouldn't they be using it? The nature of intelligence agencies demands a certain degree of secrecy/anonymity and conjecture. One author of the fourth paper, though never directly researching military or government applications, moved from the U.S.…

> The references say they can do this, and have been interested in doing it, so why wouldn't they be using it? I'm not trying to split hairs. It is possible that, despite the interest, the technology isn't at a state where it would actually be useful in practice. However, your new citation is stronger.

> It is possible that, despite the interest, the technology isn't at a state where it would actually be useful in practice.

Well, given that unreliable interrogation techniques are pretty commonly used historically, maybe it's more of a value thing.

i.e. water-boarding is unreliable and cheap, fMRI is unreliable and expensive.

Re: Machine translation of cortical activity to text with encoder–decoder framework

#90

Post by salawat »

IdiocyInAction wrote: post #76
salawat wrote: post #69

Earlier quoted context omitted.

It's a social crisis waiting to happen, especially if you can end up decoding more than just what someone uses their speech centers to articulate. This is invasive to the extreme, and seems to open the door for violations of people's intimate thoughts down the road. You may not think about it much now, but if you pay any attention to things like intrusive thoughts, or even have to deal with carefully maintaining a pu…

> Before anyone points out that the same moral outrage probably occurred with the printing press; there is a big damn difference between changing someone's mind through pamphlets, and having a direct link to the limbic system to tickle on a whim. I've recently read a short story from Ted Chiang likened the development of writing to a fundamental cybernetic enhancement of the brain. I found it to be quite enlightening…

Targeted advertising in my lifetime has gone from staging ads for kitchen appliances on daytime TV channels or in magazines for housewives to a bunch of friends getting together with their cell phones, talking about a one off odd topic, and then finding ads pop up on that topic in the next week.

To clarify: we've gone from general audience profiling, to employment of broadband sensors for surreptitious collection of data from which to make ad serving decisions. There also exist patents for installing microphones for responding to user's screaming a brand name at a TV to skip a commercial, and the practice of frame sampling of viewed content from SmartTV's. These intrusions into personal privacy come purely for the benefit of forwarding the interests of these ad servers, which also creates a vulnerability in terms of the fact that your digital footprint is available to anyone else interested in paying or requesting to be able to use it. You can't have that granular ad targeting without implementation of further surveillance capabilities.

Furthermore, there are additional consequences in that filter bubbles are created. Without you being aware, the advertising industry by default will attempt to skew your overall experience toward what they think you want to see rather than what is actually out there, or what you ask for. These algorithms allowed to run unchecked, without instilling an innoculative knowledge of the tendency of these systems to shepherd one right off the reservation given enough time, leads to things where we throw around phrases likening our society to being "post-truth", and have actually recorded multiple instances of widespread population level sentiment engineering.

So we"e garbage binned any semblance of common worldview, and invited Orwellian tiers of data collection into our lives so that other people can stand a chance at maybe serving us an ad we weren't even actively looking for in the hopes of modifying our behavvior to make a purchase happen so that they can generate revenue off of our eyeballs and content creation.

Make no mistake. Targeted advertising is a blight. It's one of those things that sounds reasonable, innocent, and possibly even helpful on the surface; but quickly sours once you start digging into the details that make it happen.

I understand some people may feel they get value out of such an arrangement; that having that ad pop up at that time genuinely makes their life easier. I ask the followi ng, however: has an ad ever taught you anything that dedicated research, and purposeful exercise of your will to purchase couldn't teach you? Has your experience searching and trying to share information online not been adversely effected in that all people's searches of the same terms have no real consistent base anymore? The answer for me in both cases is "no". Throw in the fact that if I don't regularly clean out every last trace of client side state, my wanderings through cyberspace are painstakingly mapped and integrated by an industry hell bent on coaxing Every last shred of potential value out of my mere existence with no regard for the dangers of accumulating all that data in one place.

Nowadays, you have rumblings that we should be using these technical solutions as the basis of social/political policy, and half the people making the assertion one way aren't looking at the whole picture.

I don't want the world to time-freeze at early 2000's technology by a long shot. Let me be clear on that. I do however, believe we need to seriously take a look at our capabilities, and work on creating a cohesive, widespread set of ethic/moral dicta that jive with what we purport our most valued cultural aspects are as a society. Yes, I understand that may mean converging to things I don't agree with; and that's fine. I just want as many people as possible to have the whole picture; and I don't think that right now that is actually the case.

Also, see the information warfare post from a sibling poster. Information, and tactically imposed voids of information are just as weaponizable as any object. Over longer timescales, no doubt. Still viable though.

Post reply on HN