Could It Be? Spooky Experiments That 'See' The Future • Hacker News Board

Re: Could It Be? Spooky Experiments That 'See' The Future

#121

Post by BrandonM » Wed, Jan 05, 2011, 9:43 PM UTC

gfodor wrote: post #14

bluekeybox wrote: post #5

Extraordinary claims require extraordinary evidence. The classic 5% (or even 0.1%) statistical threshold is sometimes not enough. See here for an easy-to-understand example of why that is the case: http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_... In other news, capital punishment has been installed for science journalists publishing articles that contain a question in the title that can be succinctly a…

Can someone please, please explain this to me? I've never understood why the oft stated line "extraordinary claims require extraordinary evidence" is anything other than a clever saying. Why should it be that things that follow your intuition require any less rigor to prove than those that do not, and vice versa? Presumably there should be no subjectivity to cold hard science; evidence is evidence, and a certain quan…

Since we're talking about extraordinary claims, let's examine a claim that is definitely not true, but very interesting. If we run a study with threshold p=0.05, there is a 1 in 20 chance that we will erroneously report the claim to be true.

Now, let's say ten different scientists are interested in this claim, and they're all going to run their own experiments. The chance that all ten will run an experiment with each reporting "false" is under 60%.[0] Over 40% of the time, at least one scientist will falsely conclude the existence of the phenomenon that definitely does not exist. This is an effect of running multiple independently-considered experiments without aggregating the results.

That's the Bayesian problem that people mention. Another problem entirely comes from which results will tend to get published.

Now let's consider the effect of publishing bias. Let's assume that only 20% of the scientists will attempt to publish their results regardless of the outcome, but they will always try to publish if the (false) phenomenon is shown to exist. This effect alone results in 21% of submissions being incorrect,[1] even though an incorrect result only has 5% likelihood.

Let's additionally assume that a journal will publish a false-but-interesting result 50% of the time, and the true-but-ho-hum result only 10% of the time. The final effect is that 50% of published results for this extraordinary-but-false phenomenon incorrectly report the phenomenon to be true.

Tweak the numbers all you want, but the effects of running multiple independently-considered trials, along with biased publishing, means that we are surprisingly likely to publish false conclusions.

Notes:

[0] 0.95^10 = 0.5987

[1] 0.95 * 0.2 = 0.19; 0.05 / (0.19 + 0.05) = 0.21

[2] (0.21 * 0.5) / (0.21 * 0.5 + 0.79 * 0.1) = 0.50

Re: Could It Be? Spooky Experiments That 'See' The Future

#122

Post by dhimes » Thu, Jan 06, 2011, 12:43 AM UTC

marze wrote: post #118

dhimes wrote: post #105

Earlier quoted context omitted.

The problem is that these people are allowed to vote and hold office.

And have been for millennia, deal with it.

I try... with education.

Re: Could It Be? Spooky Experiments That 'See' The Future

#123

Post by Confusion » Fri, Jan 07, 2011, 6:00 PM UTC

dandelany wrote: post #88

DanielBMarkham wrote: post #42

Earlier quoted context omitted.

Extraordinary claims require extraordinary evidence I agree, but we need to be really clear about what the claims are. It could be that there is a reproducible 1% "mystery" effect that works from future to past, but only in experiments like this. In which case claim wouldn't be extraordinary, it'd just be something we can't understand. Remember that he's still in the data gathering stage. If -- and it's a big if -- t…

> It could be that there is a reproducible 1% "mystery" effect that works from future to past, but only in experiments like this. In which case claim wouldn't be extraordinary, it'd just be something we can't understand. Erm, no, that would be pretty damn extraordinary. We know of nothing else in the universe that acts like this.

Erm, no, we actually know a lot of things that like this, with 'this' being 'something we can't explain' (talk of sensing or affecting the future is premature speculation on the explanation and irrelevant). To name one: the speed of the Voyager spacecraft. There are a zillion unexplained effects like that one and this one.