fMRI software bugs could upend years of research • Hacker News Board

Re: fMRI software bugs could upend years of research

#151

Post by shmel » Tue, Jul 05, 2016, 7:00 AM UTC

Tomte wrote: post #137

hammock wrote: post #93

Earlier quoted context omitted.

What are the actionable insights that have come out of fMRI studies? Even when properly conducted (no false positives), the conclusions that are often drawn have always felt dubious to me. Basically you are looking for regions of the brain that light up with various stimuli. Except that's as far as it goes, we don't yet understand much beyond that. It's as if you figure out that your car is making a funny sound, and…

I was participant in a study in linguistics [1] that compared native Polish and native German speakers who were put into an fRMI and played speech sounds. Both Polish and German ones. It clearly showed that speech sounds from your native language are processed in a different part of the brain than non-native speech sounds. Yes, that does not explain much. But it leads towards all kinds of questions. And I found that…

A small correction: cited study used MEG, not fMRI, as modality.

The conclusion itself doesn't look very surprising for me. We already know that the sound processing in general is the same in both hemispheres while the speech processing is very lateralized. From the continuity I could say that there should be a border where speech-like sounds sound like speech and therefore they are processed differently between hemispheres. This study seems to estimate this border.

Re: fMRI software bugs could upend years of research

#152

Post by vanderZwan » Tue, Jul 05, 2016, 7:00 AM UTC

maweki wrote: post #143

"and along the way they swipe the fMRI community for their “lamentable archiving and data-sharing practices” that prevent most of the discipline's body of work being re-analysed." That's quite funny. My girlfriend recently finished her master's thesis on data sharing for neuroscience data and created a model for universal access to research data across institutions, but came to the conclusion that making researchers…

Meanwhile, the research I work for publishes all of their data. In fact, the main reason I have been hired by their professor as a programmer/interaction designer (through HN no less) is because the next project will produce a mountain of data and he does not want it to collect dust after the project is done. I am supposed to make the data even more accessible to other researchers through a web-interface.

Aside from the ethical motivations, other benefits are pretty obvious: there is at least one postdoc in the research department that I know of was recruited because he found a completely different use for the data shared in the past and published a paper based on his own analysis.

(The research group works with mouse brains so privacy is not an issue)

Re: fMRI software bugs could upend years of research

#153

Post by 50CNT » Tue, Jul 05, 2016, 7:08 AM UTC

nerdponx wrote: post #142

jballanc wrote: post #95

Earlier quoted context omitted.

Not to mention the researchers' and post-docs' and students' time. By not paying to store it, you're risking the possibility that all of this money is essentially flushed down the toilet by a software bug because it cannot now be reanalyzed.

Think of the flipside: by paying to store it you are paying to run the risk that your research will be invalidated.

Whilst the incentives are all wrong in this situation, that's a good thing, right?

Re: fMRI software bugs could upend years of research

#154

Post by a_bonobo » Tue, Jul 05, 2016, 7:34 AM UTC

vanderZwan wrote: post #152

maweki wrote: post #143

"and along the way they swipe the fMRI community for their “lamentable archiving and data-sharing practices” that prevent most of the discipline's body of work being re-analysed." That's quite funny. My girlfriend recently finished her master's thesis on data sharing for neuroscience data and created a model for universal access to research data across institutions, but came to the conclusion that making researchers…

Meanwhile, the research I work for publishes all of their data. In fact, the main reason I have been hired by their professor as a programmer/interaction designer (through HN no less) is because the next project will produce a mountain of data and he does not want it to collect dust after the project is done. I am supposed to make the data even more accessible to other researchers through a web-interface. Aside from…

I guess it depends on research field and journal - in biology most journals require you to deposit your raw genomic reads to the Sequence Read Archive, this is still sometimes missed by editors and reviewers.

Random example for the requirement of data depositing from Journal of Plant Physiology: "If new sequence data are reported, insert the following: “Sequence data from this article have been deposited at XXX under accession number(s) YY000000.‘‘ http://cdn.elsevier.com/promis_misc/jpp_Instruction.pdf

Another random example from the Nature Group http://www.nature.com/authors/policies/availability.html

AFAIK only few funding bodies require the deposit of raw data...

Re: fMRI software bugs could upend years of research

#155

Post by Retric » Tue, Jul 05, 2016, 9:59 AM UTC

slowmovintarget wrote: post #145

Retric wrote: post #111

Earlier quoted context omitted.

The logic is odd but strait forward. Basically, free will requires people to make choices independent of the past state of the universe. Aka if you replayed to universe up to this point several times they would not always respond in the same way. If that's the case people should be able to make unpredictable choices by definition. However, when asked to be unpredictable they fail. So, at best people have a limited fo…

Quick, don't think of an elephant. There are many things that, when asked, people are suddenly bad at doing. I don't think that's evidence.

That I had no problem doing. What's the point?

Re: fMRI software bugs could upend years of research

#156

Post by bad_user » Tue, Jul 05, 2016, 10:00 AM UTC

Retric wrote: post #111

bad_user wrote: post #94

Earlier quoted context omitted.

I've never heard of that argument, though I admit I'm new to this discussion. But, given that people can and have built pretty good random generators, by finding sources of entropy and even with pure math, thus sidestepping our limitations, the point is kind of moot IMHO.

The logic is odd but strait forward. Basically, free will requires people to make choices independent of the past state of the universe. Aka if you replayed to universe up to this point several times they would not always respond in the same way. If that's the case people should be able to make unpredictable choices by definition. However, when asked to be unpredictable they fail. So, at best people have a limited fo…

Well OK, but if you want nondeterministic behavior, you can find a source of entropy and make decisions based on a random number (e.g. flip a coin). You chose to do it and it invalidates the hypothesis, because if you replay the universe up to this point, you won't behave in the same way and the choices you just made by flipping a coin are independent of the past state of the universe.

So what am I missing, as I feel we are going into non-falsifiable territory.

Re: fMRI software bugs could upend years of research

#157

Post by Tomte » Tue, Jul 05, 2016, 10:04 AM UTC

shmel wrote: post #151

Tomte wrote: post #137

Earlier quoted context omitted.

I was participant in a study in linguistics [1] that compared native Polish and native German speakers who were put into an fRMI and played speech sounds. Both Polish and German ones. It clearly showed that speech sounds from your native language are processed in a different part of the brain than non-native speech sounds. Yes, that does not explain much. But it leads towards all kinds of questions. And I found that…

A small correction: cited study used MEG, not fMRI, as modality. The conclusion itself doesn't look very surprising for me. We already know that the sound processing in general is the same in both hemispheres while the speech processing is very lateralized. From the continuity I could say that there should be a border where speech-like sounds sound like speech and therefore they are processed differently between hemi…

Thanks for the correction!

I misremembered because my professor's group did a lot of fMRI stuff, as well, and in the seminars we mostly talked about those.

Speech/language and brain is fascinating. There are resident linguists at major hospitals who are consulted before neurosurgery. Speech sounds are processed faster than other sounds in our brain. Rearranging sentences from active voice to passive voice, silently in your head, lead to easily seen activity in fMRIs, distinct from non-linguistic mental actions. And so on.

Re: fMRI software bugs could upend years of research

#158

Post by UVDMAS » Tue, Jul 05, 2016, 11:52 AM UTC

The paper has been rebutted by other researchers who argue that the original results hold:

"This technical report revisits the analysis of family-wise error rates in statistical parametric mapping - using random field theory - reported in (Eklund et al., 2015). Contrary to the understandable spin that these sorts of analyses attract, a review of their results suggests that they endorse the use of parametric assumptions - and random field theory - in the analysis of functional neuroimaging data. We briefly rehearse the advantages parametric analyses offer over nonparametric alternatives and then unpack the implications of (Eklund et al., 2015) for parametric procedures."

http://arxiv.org/abs/1606.08199

Re: fMRI software bugs could upend years of research

#159

Post by Houshalter » Tue, Jul 05, 2016, 1:11 PM UTC

threatofrain wrote: post #72

Strilanc wrote: post #61

Earlier quoted context omitted.

Why would this put an end to that line of reasoning? I'd expect it to flare up the debate, not end the debate. They didn't disprove behavior being computed, they demonstrated that a class of data supporting it was useless. The natural reaction to this isn't "okay we give up", it's "better go get some good data". (Also, I don't like mixing the question "Is our behavior computed?" with the question "Does computed behav…

The problem with free will debates has to do with its definition. I think once people start hammering out the definition, free will either becomes a wimpy variant that people didn't want to talk about in the first place, or it becomes so ambitious that it's outside the scope of science to discuss. If you define free will as the ability to generate and choose options based on constraints, then people get bored of that…

This has little to do with the study in question. It wasn't about weird metaphysics, it was that MRI scans could predict what choice people would make, before they reported making a choice. Showing that the unconscious mind makes choices before our conscious mind is even aware of it. I don't think this result is affected by this new issue, because it seems to have been done with old fashioned EEGs too.

You can read more about this stuff here: http://io9.gizmodo.com/5975778/scientific-evidence-that-you-...

This adds to other work from split brain patients, that our right brain explains away the choices we've made even if those explanations are totally false. E.g. they ask the left brain to pick up a toy soldier, and ask the right brain why they did that. They say "well because I always liked toy soldiers when I was a kid", or some other made up explanation. There's even cases where one side of the brain is paralyzed and can't move its arm, and the other side makes up explanations why it doesn't want to move its arm and refuses to believe it's paralyzed.

I think it's possible humans don't have Free Will. Not just in a philosophical, or determinism vs nondeterminism sense, but in a very practical sense. That our actions are highly predictable. And that once you start to see the inner workings of the machine that is our minds, it starts to seem a whole lot less magical.

This intuition is hard to explain, but in general many systems seem to have "agency" until you understand how they work, and then they start to seem just like normal "non agency" things. As we learn more about how humans work, we start to look a lot less agenty. More on that here: http://lesswrong.com/lw/mb0/agency_is_bugs_and_uncertainty/

Re: fMRI software bugs could upend years of research

#160

Post by omginternets » Tue, Jul 05, 2016, 1:37 PM UTC

Trombone12 wrote: post #62

Well, I think the problems with interpreting fMRI scans have been at least vaguely known since that time a dead salmon activated its neurons when asked to judge the of a person from a photo, this was in 2009. http://www.wired.com/2009/09/fmrisalmon/

I don't see how that's really relevant. The point of the dead salmon paper was "do your damn FWE corrections". The point of paper, is rather "popular tools make unsound assumptions" (or, inversely, "researchers are violating the assumptions made by their tools").

The problem in the salmon paper is easy to fix. The problem in this paper is trickier.