Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

fMRI software bugs could upend years of research

theregister.co.uk

Post reply on HN
Print view View original

121–130 of 192 posts

Re: fMRI software bugs could upend years of research

#121

Post by glial »

bpchaps wrote: post #103
jballanc wrote: post #78

The real takeaway lesson from this research should be the vital importance of Open Data to the modern scientific enterprise: > "lamentable archiving and data-sharing practices" that prevent most of the discipline's body of work being re-analysed. Keeping data private before publication is (at this point in time) understandable. Once results are published, however, there is no excuse for not depositing the raw data in…

Yeah, it's pretty frustrating. I tried getting my own data from an fMRI study, but was told that the signed paperwork specifically disallowed this sort of thing. Not even my doctor could request it. The only option I have is to completely withdrawal myself from the study, but that would be a pretty dick move. The other option is to wait until next year when the study's wrapped and then request it. Though, I'm not eve…

I was able to get a copy of my data from a MEG study, supposedly because it was part of my electronic medical record so I was entitled to it. Not sure whether that was sound reasoning, but it got the result I wanted.

Re: fMRI software bugs could upend years of research

#122

Post by AlexCoventry »

nerdponx wrote: post #117

"Further: “Our results suggest that the principal cause of the invalid cluster inferences is spatial autocorrelation functions that do not follow the assumed Gaussian shape”." This has nothing to do with bugs and everything to do with bad statistical analysis. It's Google Flu all over again.

It's all relative. At the right level of abstraction, bad statistical analysis is a bug.

Re: fMRI software bugs could upend years of research

#123

Post by neoeldex »

threatofrain wrote: post #63
randcraw wrote: post #44

As someone who works in the biomedical imaging business and is also a fan of philosophy, I think this news will matter more to folks in the latter camp. For a couple years now philosophers have insisted that fMRI images prove there is no such thing as free will. Today's revelation should put an end to that whole line of reasoning (and the absurd amount of fatalism that it engendered). (The back story: Apparently fMRI…

There's free will as in a decision-making system that generates different options and selects one based on constraints, which is something a computer can do, and then there's free will as in the ability to defy the mechanics of one's biophysics, which is just an indirect way of saying metaphysical soul. I don't think empiricists can even discuss the kind of free will which defies causal explanation, the kind of mind…

I don't think there's such a thing as a metaphysical soul. What would that look like anyhow? But I do think any sufficient complex system could show patterns which could be perceived as random. Although they may be hard to reason about. I believe consciousness arises from such processes, and the perception of free will.

Re: fMRI software bugs could upend years of research

#125

Post by bjourne »

Does it mean studies like these are likely bunk?

  http://kangleelab.com/articles/Paper0002_0009.pdf
  https://med.stanford.edu/news/all-news/2016/05/moms-voice-activates-different-regions-in-children-brains.html
  https://www.theguardian.com/science/2015/apr/21/babies-feel-pain-like-adults-mri-scan-study-suggests
  https://news.brown.edu/articles/2013/06/breastfeeding
And with bunk I mean doesn't show what they claim.

Re: fMRI software bugs could upend years of research

#126

Post by nonbel »

mattkrause wrote: post #115
nonbel wrote: post #102

Earlier quoted context omitted.

I think we are largely in agreement on the surface. The problem is that they are attributing the "positive" results to statistical error, when it isn't due to that. The stats are working just fine and correctly telling us the null hypothesis is false. The problem is with the crappy null hypothesis, not the stats.

The experiment is set up so that there are no true positives. When the threshold is p Are you trying to say that it's not the t-test per se, but something before it in the pipeline? If so, sure, of course, but that's not the null. The null is that chunks randomly labeled as ON are the same as chunks randomly labeled as OFF, and since it's not physically possible for the labels to affect the signal, this null must be…

>"The experiment is set up so that there are no true positives."

No, a true positive is when the null model is called false when it is false. They say the null is known to be false. The errors here are actually the false negatives, in this case that is every "non-significant" result.

>"When the threshold is pOnly if the null model is correct. In this case they describe problematic aspects of their null model. Ie they know it is false.

Re: fMRI software bugs could upend years of research

#127

Post by e12e »

jballanc wrote: post #78

The real takeaway lesson from this research should be the vital importance of Open Data to the modern scientific enterprise: > "lamentable archiving and data-sharing practices" that prevent most of the discipline's body of work being re-analysed. Keeping data private before publication is (at this point in time) understandable. Once results are published, however, there is no excuse for not depositing the raw data in…

This is medical data. "Open repository" and "medical data on individuals" doesn't really mix well. Ask the next person denied health insurance based on an open access fmri scan (Just to name the most basic/trivial example).

In fact, "simpler" things like heart rate etc, are not so simple, as mixed with other factors that are needed for control, such data can be surprisingly hard to meaningfully anonymize. I'm not saying we should give up on open medical data, but it is definitively different from eg: open data regarding a physics or chemical experiment, material science etc.

[ed: There's of course projects to collect data, but limit access - in that sense it can be "open" but one would need permission approval to work with the data. There are many such projects, like for Norway, for questionnaires and similar research: http://www.nsd.uib.no/personvern/om/english.html - research that uses standard clauses for possible re-use is also available to be combined with new studies and meta-studies. Often these data are only available in aggregate and/or anonomyzed form.]

Re: fMRI software bugs could upend years of research

#128

Post by nonbel »

mattkrause wrote: post #108
nonbel wrote: post #99

Earlier quoted context omitted.

>"You are the one who keeps saying the authors don't even realize what those assumptions are, but you haven't pointed out anything besides what the authors said." I am saying they are confused because they say "the null hypothesis should be true" under their conditions, when they know for certain that it is false! Therefore these "false positives" are not false at all. They are totally legit "true positives". These a…

Okay, why do you think the null is crappy? In the absence of any information whatsoever, the idea that an analysis pipeline produces false positives at or below its nominal rate seems pretty reasonable. But, they have some prior information. Let's look at the E1 paradigm (2 sec on, 6 sec off). In the NeuroImage Paper (Figure 1A, 2A), the FWER on voxel tests is statistically indistinguishable from 5%. In other words,…

>"Okay, why do you think the null is crappy?"

Because their goal is to determine if some sort of treatment has an effect. If the null is false for other reasons, then statistical significance can't be used to support the existence of a treatment effect. So these would be pointless, pedantic calculations.

Re: fMRI software bugs could upend years of research

#129

Post by roywiggins »

clord wrote: post #112
acqq wrote: post #56

Earlier quoted context omitted.

Free will as in "there's something that can't be explained by chemistry and electrical signals in the brain" is just an attempt of some believers to rationalize their belief in the kind of the supernatural "soul."

Are you suggesting there is no room for quantum effects in the brain? Honest question because I thought determinism is out of fashion now-a-days for that reason.

Quantum effects are still physical. If your behavior is determined by the collapse of a wavefunction, is that more free-willish than behavior determined by a prior state? A double-slit photon doesn't choose a slit via free will, so I'm not sure how quantum mechanics helps inject free will into the brain.

Re: fMRI software bugs could upend years of research

#130

Post by [deleted] »

randcraw wrote: post #44

As someone who works in the biomedical imaging business and is also a fan of philosophy, I think this news will matter more to folks in the latter camp. For a couple years now philosophers have insisted that fMRI images prove there is no such thing as free will. Today's revelation should put an end to that whole line of reasoning (and the absurd amount of fatalism that it engendered). (The back story: Apparently fMRI…

[deleted]
Post reply on HN