http://norvig.com/experiment-design.html
here on HN again. How well does Bem's set of experiments hold up?
81–90 of 123 posts
http://norvig.com/experiment-design.html
here on HN again. How well does Bem's set of experiments hold up?
This is trivial software to write. Why does someone not quickly whip up a web app that does the picture thing as described in the experiment, then we can personally test if we have ESP senses or not? Specs: Two buttons - ESP mode or non-ESP mode. In non-ESP mode, 60 random pictures are shown and we are to guess. Then it gives the correct one. In ESP mode, add some porn. If the results are different, then we have ESP.…
If we repeated the experiment here, publicly, we'd all know whether we were in the control or experiment group just about immediately. The results wouldn't be valid, would they?
As such, test subjects knowing they are in the group with the erotic pictures should still be able to beat the 50% threshold.
Edit: Actually, I'm reading through the actual experiment and it appears all 100 sessions used both erotic and nonerotic pictures of varying arousal value. Also, both the position of the picture and the picture itself were not actually chosen by the computer until after the test taker made the choice, although they were told differently, making it a test for a future event. So, yes, I think we would already be compromised for trying to recreate the test.
Earlier quoted context omitted.
If you're going to use highly subjective frequentist statistics at all, p minimum gold standard for extraordinary claims. If the phenomenon is real, and not bad statistics, it only requires two and a half times as many subjects to get p < 0.001 instead of p < 0.05. Physicists, who don't want to have to put up with this crap, use p < 0.0001. p < 0.05 is asking for trouble.
Ok, then let's funding psychology like we fund physics. I would love to run 1000+ patient studies to test psychotherapies, and in fact we'd be able to answer some really interesting questions if we did, but there is currently no way of doing this.
And I suspect the ultimate reason it's not done this way... is that scientists in certain fields would publish a lot fewer papers, not slightly fewer but a lot fewer, if all the effects they were studying had to be real.
Here's one IEEE paper: "a perceptual channel for information transfer over kilometer distances - historical perspective and recent research" http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=1454...
Can't find the online version anymore, so here's the version I found (9.5MB pdf): http://www.box.net/shared/inxg0nld9r
Earlier quoted context omitted.
Ok, then let's funding psychology like we fund physics. I would love to run 1000+ patient studies to test psychotherapies, and in fact we'd be able to answer some really interesting questions if we did, but there is currently no way of doing this.
I repeat, you do not need 1000 times as many subjects to get results that are 1000 times as significant! If 40 subjects gets you results with p all the published results being real effects, instead of most of them failing to replicate when tested, sounds like a great tradeoff to me. And I suspect the ultimate reason it's not done this way... is that scientists in certain fields would publish a lot fewer papers, not s…
"Extraordinary claims require extraordinary evidence" -- and I wouldn't count these small studies as extraordinary. Unfortunatley, publishing these kind of claims prematurely help the more gullible among us to fall for ridiculous claims from psychics and others who would take advantage of them. (The authors of "The Secret", I'm looking at you.) Commenters on NPR's website (not exactly the dumbest audience online) hav…
Extraordinary claims require extraordinary evidence I agree, but we need to be really clear about what the claims are. It could be that there is a reproducible 1% "mystery" effect that works from future to past, but only in experiments like this. In which case claim wouldn't be extraordinary, it'd just be something we can't understand. Remember that he's still in the data gathering stage. If -- and it's a big if -- t…
Erm, no, that would be pretty damn extraordinary. We know of nothing else in the universe that acts like this.
Earlier quoted context omitted.
If we repeated the experiment here, publicly, we'd all know whether we were in the control or experiment group just about immediately. The results wouldn't be valid, would they?
For the "Who's got the porn" experiment I don't think test takers knowing which group they were in would make much difference. My understanding is that both groups would be told they were being tested for ESP. However, one group is told hot stimulating pictures are mixed in with regular pictures, and the other group is told there are only regular pictures. According to the article, the group with no added incentive (…
So it looks like I'll be studying my Mega Millions picks tonight. My methodology: 1. Buy ticket 2. Go AWOL until after the drawing. 3. Study numbers 4. Look up results 5. Profit