[deleted]
fMRI software bugs could upend years of research
81–90 of 192 posts
Re: fMRI software bugs could upend years of research
#82The real takeaway lesson from this research should be the vital importance of Open Data to the modern scientific enterprise: > "lamentable archiving and data-sharing practices" that prevent most of the discipline's body of work being re-analysed. Keeping data private before publication is (at this point in time) understandable. Once results are published, however, there is no excuse for not depositing the raw data in…
Uh, where would you store it though? IIRC from the time a relative was going through a chemistry PHD, they produced several GBs of raw data every half hour . Storage is cheap but it's not that cheap...
Even storing - and making accessible - a fraction of it would be a drastic improvement over the current status quo.
Re: fMRI software bugs could upend years of research
#83[deleted]
Re: fMRI software bugs could upend years of research
#84The dead salmon study seems relevant here in discussion of how fMRI is used, especially the theory -ladenness of observations.
http://blogs.scientificamerican.com/scicurious-brain/ignobel...
Re: fMRI software bugs could upend years of research
#85As someone who works in the biomedical imaging business and is also a fan of philosophy, I think this news will matter more to folks in the latter camp. For a couple years now philosophers have insisted that fMRI images prove there is no such thing as free will. Today's revelation should put an end to that whole line of reasoning (and the absurd amount of fatalism that it engendered). (The back story: Apparently fMRI…
So have these studies been invalidated by the software bug as well? If so, do you have any pointers? I. e. which were the infamous studies, and did they indeed use the faulty software to derive their conclusions? I'm genuinely interested.
http://www.theatlantic.com/magazine/archive/2016/06/theres-n...
This article preceded awareness of the bug.
Re: fMRI software bugs could upend years of research
#86As someone who works in the biomedical imaging business and is also a fan of philosophy, I think this news will matter more to folks in the latter camp. For a couple years now philosophers have insisted that fMRI images prove there is no such thing as free will. Today's revelation should put an end to that whole line of reasoning (and the absurd amount of fatalism that it engendered). (The back story: Apparently fMRI…
Well, the non-pop philosophers will understand that as a minor hit to the model of consciousness as solely concerned with post-facto rationalization and basically removed from the real-time decision making loop. Those studies were never strong evidence anyway, the protocols were pretty weak as they required asking the subjects to self-report at what time they "decided" to act. The real free will debate is about both…
Interestingly one of the best arguments apposed to free will is people being terrible random number generators. Which tends to be hand waved away.
Which suggests people have already decided which side they believe in and only try and rationalize it after the fact. ;)
Re: fMRI software bugs could upend years of research
#87Doesn't sound like a straight up bug, but rather unsound statistical methods which can happen with or without software. You get the same problem with finite element analysis software: the operator has to be aware of all the assumptions baked in, and has to ensure that the input conforms to them.
I'd say it's a bug, since the unsound statistical methods are incorporated into the three most common software packages that are specifically intended to be used for fMRI analysis. If these methods don't work well for fMRI data, fMRI software shouldn't be using them. From the paper: "Using this null data with different experimental designs, we estimate the incidence of significant results. In theory, we should find 5…
No, a statistical modeling assumption is not the same as a software bug. There is no perfect model, and nonparametric tests are approximations at best. The important thing is that the publication reports the modeling assumption so that readers can judge the conclusions relative to that choice. Now we know to judge results using the Gaussian model to have a high false positive rate.
On the other hand, no researcher would knowingly publish results affected by software bugs, e.g. null pointers or arithmetic errors... hopefully.
Re: fMRI software bugs could upend years of research
#88As someone who works in the biomedical imaging business and is also a fan of philosophy, I think this news will matter more to folks in the latter camp. For a couple years now philosophers have insisted that fMRI images prove there is no such thing as free will. Today's revelation should put an end to that whole line of reasoning (and the absurd amount of fatalism that it engendered). (The back story: Apparently fMRI…
The explanation I heard about that is that recognizing that we made a choice happens after we actually make the choice.
Re: fMRI software bugs could upend years of research
#89So people don't perform complex, goal-focused motor tasks without having a goal ahead of time after all.
Wow, philosophy people.
EDIT: Cry about it all you want. It won't change the fact that in 100 years people will look back and wonder if an entire academic discipline was afflicted with some form of literal mental retardation.
Re: fMRI software bugs could upend years of research
#90"Our results suggest that the principal cause of the invalid cluster inferences is spatial autocorrelation functions that do not follow the assumed Gaussian shape." In other words, researchers cut corners. You should never assume that something is a certain way without rigorously proving it. How did these papers make it past peer review?
I reckon there is a vast number of similar problems in other studies across most fields. Linear regression, ANOVA, and t-tests are widely used techniques with an assumption of Gaussian errors that is rarely checked. I wonder how many of these papers would become null results if switched to nonparametric tests...