Live data from Hacker News

fMRI software bugs could upend years of research

theregister.co.uk

11–20 of 192 posts

Re: fMRI software bugs could upend years of research

#11
post #9

"Our results suggest that the principal cause of the invalid cluster inferences is spatial autocorrelation functions that do not follow the assumed Gaussian shape." In other words, researchers cut corners. You should never assume that something is a certain way without rigorously proving it. How did these papers make it past peer review?

n=30 ought to be enough for anybody

n>=30 is not the same as gaussian distribution

Re: fMRI software bugs could upend years of research

#15

Doesn't sound like a straight up bug, but rather unsound statistical methods which can happen with or without software. You get the same problem with finite element analysis software: the operator has to be aware of all the assumptions baked in, and has to ensure that the input conforms to them.

I'd say it's a bug, since the unsound statistical methods are incorporated into the three most common software packages that are specifically intended to be used for fMRI analysis. If these methods don't work well for fMRI data, fMRI software shouldn't be using them.

From the paper: "Using this null data with different experimental designs, we estimate the incidence of significant results. In theory, we should find 5% false positives (for a significance threshold of 5%), but instead we found that the most common software packages for fMRI analysis (SPM, FSL, AFNI) can result in false-positive rates of up to 70%." (http://www.pnas.org/content/early/2016/06/27/1602413113.full)

Choosing a bad algorithm for your software is as much of a bug as a null-pointer crash, and is something that needs to be tested for when building software.

Re: fMRI software bugs could upend years of research

#16
Just goes to show when you're doing science you need to test and validate your experimental methodology, including the tools you use. In computer vision, its common to need to do some kind of calibration for many algorithms which can usually reveal some kind of statistical error or problem. I wonder why none of the researchers thought to do some very simple validation of the data?

And I wonder if the software was at one point correct and then this bug was introduced at a later point? Many times it feels like after a company does a formal scientific validation they never do it again despite the fact they have engineers constantly working on the code...

Re: fMRI software bugs could upend years of research

#18
post #7

It's already been known for several years that almost all MRI brain scan research is wrong, what exactly is new here?

> almost all MRI brain scan research is wrong Source for a layman?

I wasn't in fmri research but some fellow students in my lab were on it. I do know there was a study with dead salmon which were showing up as having "active" brain regions. I have to find it but am on my phone.

It's easy to imagine the main difficulty with this is mapping signal to actual thoughts and regions. It's a really complex biology and physics to reduce.

Re: fMRI software bugs could upend years of research

#20

Earlier quoted context omitted.

> almost all MRI brain scan research is wrong Source for a layman?

I wasn't in fmri research but some fellow students in my lab were on it. I do know there was a study with dead salmon which were showing up as having "active" brain regions. I have to find it but am on my phone. It's easy to imagine the main difficulty with this is mapping signal to actual thoughts and regions. It's a really complex biology and physics to reduce.

what is your field? Neuroscience?
Post reply on HN