Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

fMRI software bugs could upend years of research

theregister.co.uk

Post reply on HN
Print view View original

141–150 of 192 posts

Re: fMRI software bugs could upend years of research

#141

Post by nonbel »

mattkrause wrote: post #132
nonbel wrote: post #126

Earlier quoted context omitted.

>"The experiment is set up so that there are no true positives." No, a true positive is when the null model is called false when it is false. They say the null is known to be false. The errors here are actually the false negatives, in this case that is every "non-significant" result. >"When the threshold is p Only if the null model is correct. In this case they describe problematic aspects of their null model. Ie the…

> They say the null is known to be false. Can you quote where they say that, because the article says: "Because two groups of subjects are randomly drawn from a large group of healthy controls, the null hypothesis of no group difference in brain activation should be true. Moreover, because the resting-state fMRI data should contain no consistent shifts in blood oxygen level-dependent (BOLD) activity, for a single gro…

The key point is that I am saying the second quote does say the null is false. This is where we differ. The actual null hypothesis they are testing is different than the one they actually care and talk about. It includes additional auxilliary assumptions that they say are wrong.

What gets plugged in as the null hypothesis is not a problem in the realm of statistics. That is a choice of the researcher. The stats algorithm worked fine.

Re: fMRI software bugs could upend years of research

#142

Post by nerdponx »

jballanc wrote: post #95
zachrose wrote: post #92

Earlier quoted context omitted.

But how expensive was the lab and the instruments that produce that amount of data?

Not to mention the researchers' and post-docs' and students' time. By not paying to store it, you're risking the possibility that all of this money is essentially flushed down the toilet by a software bug because it cannot now be reanalyzed.

Think of the flipside: by paying to store it you are paying to run the risk that your research will be invalidated.

Re: fMRI software bugs could upend years of research

#143

Post by maweki »

"and along the way they swipe the fMRI community for their “lamentable archiving and data-sharing practices” that prevent most of the discipline's body of work being re-analysed."

That's quite funny. My girlfriend recently finished her master's thesis on data sharing for neuroscience data and created a model for universal access to research data across institutions, but came to the conclusion that making researchers share their data is a bigger hurdle than actually implementing the system.

The main reason for lack of sharing, she postulated, is, that studies (that create funding for the researcher who publishes them) can be done using just the raw data and researchers who create data want to publish all the studies/papers for themselves (because "they" paid for the data acquisition) and are also afraid to publish underlying data for it to be harder for others to falsify their results, which would lead, in their opinion, to funding going away.

Edit: of course there are privacy issues for the test subjects as well.

Re: fMRI software bugs could upend years of research

#144

Post by kriro »

maweki wrote: post #143

"and along the way they swipe the fMRI community for their “lamentable archiving and data-sharing practices” that prevent most of the discipline's body of work being re-analysed." That's quite funny. My girlfriend recently finished her master's thesis on data sharing for neuroscience data and created a model for universal access to research data across institutions, but came to the conclusion that making researchers…

Please ask her to keep working on the problem and keep fighting the good fight. I think it's a political problem. If research grants were only given out if all data was made available (and while we're at it open source all the code produced) we'd be a huge step further down the right path.

There is very, very strong resistance to these sort of ideas. The idealist in me likes to think that there shouldn't be much resistance after all scientists should be happy if their results get double checked (for free!). I can only assume that a lot of people are very afraid of having their data double checked (or possibly never had data in the first place). Additionally access to data is a competitive advantage as it's often hard to generate.

Re: fMRI software bugs could upend years of research

#145

Post by slowmovintarget »

Retric wrote: post #111
bad_user wrote: post #94

Earlier quoted context omitted.

I've never heard of that argument, though I admit I'm new to this discussion. But, given that people can and have built pretty good random generators, by finding sources of entropy and even with pure math, thus sidestepping our limitations, the point is kind of moot IMHO.

The logic is odd but strait forward. Basically, free will requires people to make choices independent of the past state of the universe. Aka if you replayed to universe up to this point several times they would not always respond in the same way. If that's the case people should be able to make unpredictable choices by definition. However, when asked to be unpredictable they fail. So, at best people have a limited fo…

Quick, don't think of an elephant.

There are many things that, when asked, people are suddenly bad at doing. I don't think that's evidence.

Re: fMRI software bugs could upend years of research

#146

Post by brandonl222 »

maweki wrote: post #143

"and along the way they swipe the fMRI community for their “lamentable archiving and data-sharing practices” that prevent most of the discipline's body of work being re-analysed." That's quite funny. My girlfriend recently finished her master's thesis on data sharing for neuroscience data and created a model for universal access to research data across institutions, but came to the conclusion that making researchers…

It's a rather sad reflection of society, no telling how may breakthroughs we're missing out on due to the lack of cooperation in science on this front.

Re: fMRI software bugs could upend years of research

#147

Post by 50CNT »

slowmovintarget wrote: post #145
Retric wrote: post #111

Earlier quoted context omitted.

The logic is odd but strait forward. Basically, free will requires people to make choices independent of the past state of the universe. Aka if you replayed to universe up to this point several times they would not always respond in the same way. If that's the case people should be able to make unpredictable choices by definition. However, when asked to be unpredictable they fail. So, at best people have a limited fo…

Quick, don't think of an elephant. There are many things that, when asked, people are suddenly bad at doing. I don't think that's evidence.

I've always found that these "Quick, don't think of 'x'" things are a silly example. You end up thinking of X the moment you read X, just before it gets put in context of "not" and "think". It's like telling the parser not to tokenize a given word. Then there's automaticity, which you have to turn off, which is made harder by "Quick", further setting you up to think further of elephants till you get around to grasping the meaning of the phrase.

Re: fMRI software bugs could upend years of research

#148

Post by wiz21 »

RGamma wrote: post #53
randcraw wrote: post #44

As someone who works in the biomedical imaging business and is also a fan of philosophy, I think this news will matter more to folks in the latter camp. For a couple years now philosophers have insisted that fMRI images prove there is no such thing as free will. Today's revelation should put an end to that whole line of reasoning (and the absurd amount of fatalism that it engendered). (The back story: Apparently fMRI…

To be fair, there are more arguments to be made contra free will (thought experiments like: At what point from conception to adulthood would free will develop (if at all), could free will be a merely probabilistic byproduct, etc...) and even if the details are wrong in these fMRI images, the general notion that a motor signal is generated before it appears in consciousness might still be valid.

Funny. One measure when motor signal and sees that it can measure when it appears in consciousness. Do me a favor, how can you be sure that what you measure is actually consciousness. Give me a "measurable" consciousness.

Moreover, Freud basically says that free will doesn't exist because he makes a difference between conscious and subconscious... The latter being a huge bunch of thoughts you don't make/change at will...

Moreover, predicting if a ball will fall when put above the ground is something. Predicting the next second of life of the universe is something else. So predicting a subject will raise its left/right hand is something. Predicting what will be his opinion about abortion is an entire story

All of this reminds me of studies done long ago which tried to predict if someone was a criminal or not by measuring the shape of its skull...

Re: fMRI software bugs could upend years of research

#149

Post by lostlogin »

brandonl222 wrote: post #146
maweki wrote: post #143

"and along the way they swipe the fMRI community for their “lamentable archiving and data-sharing practices” that prevent most of the discipline's body of work being re-analysed." That's quite funny. My girlfriend recently finished her master's thesis on data sharing for neuroscience data and created a model for universal access to research data across institutions, but came to the conclusion that making researchers…

It's a rather sad reflection of society, no telling how may breakthroughs we're missing out on due to the lack of cooperation in science on this front.

There is also the motivations of the scanner owner to consider. They are paid to do scans. A cheap scan might be low hundreds of dollars per hour. A more commercial rate would be well into the thousands. We don't want to dry up now do we?

Re: fMRI software bugs could upend years of research

#150

Post by lostlogin »

hammock wrote: post #93
randcraw wrote: post #44

As someone who works in the biomedical imaging business and is also a fan of philosophy, I think this news will matter more to folks in the latter camp. For a couple years now philosophers have insisted that fMRI images prove there is no such thing as free will. Today's revelation should put an end to that whole line of reasoning (and the absurd amount of fatalism that it engendered). (The back story: Apparently fMRI…

What are the actionable insights that have come out of fMRI studies? Even when properly conducted (no false positives), the conclusions that are often drawn have always felt dubious to me. Basically you are looking for regions of the brain that light up with various stimuli. Except that's as far as it goes, we don't yet understand much beyond that. It's as if you figure out that your car is making a funny sound, and…

When your about to dig a chunk of someone's brain out and have a variety of routes to get to the offending region, it's handy to know what you might be able to avoid knocking off.
Post reply on HN