Bayes's Theorem: What's the Big Deal? • Hacker News Board

Re: Bayes's Theorem: What's the Big Deal?

#71

Post by DanBC » Tue, Jan 05, 2016, 6:01 PM UTC

hyperpape wrote: post #44

Anderkent wrote: post #28

Earlier quoted context omitted.

It's true that when the question is formed in frequentist terms, the answer is much more intuitive. But is that how the problem occurs in real life? The doctor doesn't see ten thousand people take a test; they see a person take a test, and get either a positive or negative result. The traditional way of forming the problem seems closer to actual experience: 'your patient tested positive. you know how accurate the tes…

I'm not quite sure what you're saying. Doctors don't observe probabilities or enormous frequencies. Either way, there are good odds that this is information that someone is communicating to them, not the result of their personal experience.

"Reckoning with risk" by Gerd Gigerenzer is interesting.

https://plus.maths.org/content/reckoning-risk

> The prescription put forward is simple. Essentially, we should all be using natural frequencies to express and think about uncertain events. Conditional probabilities are used in the first of the following statements; natural frequencies in the second (both are quoted from the book):

> The probability that one of these women [asymptomatic, aged 40 to 50, from a particular region, participating in mammography screening] has breast cancer is 0.8 percent. If a woman has breast cancer, the probability is 90 percent that she will have a positive mammogram. If a woman does not have breast cancer, the probability is 7 percent that she will still have a positive mammogram.

> Imagine a woman who has a positive mammogram. What is the probability that she actually has breast cancer?

> Eight out of every 1,000 women have breast cancer. Of these 8 women with breast cancer, 7 will have a positive mammogram. Of the remaining 992 women who don't have breast cancer, some 70 will still have a positive mammogram. Imagine a sample of women who have positive mammograms in screening. How many of these women actually have breast cancer?

Re: Bayes's Theorem: What's the Big Deal?

#72

Post by lottin » Tue, Jan 05, 2016, 6:04 PM UTC

gaur wrote: post #11

njohnson41 wrote: post #2

Good article. I'm only a bit disappointed that the author seems not to realize that Bayes' theorem is just a simple consequence of probability theory, and should be attractive not because "maybe the brain is Bayesian", but because it is based on sound set-theoretic and analytic principles. If Bayes' theorem is false, so is probability theory, and so is nearly everything we know about probability. Edit: Here is a good…

I guess a hard-line frequentist (if such a person exists) would counter that you can't assign probabilities to hypotheses or fixed parameters. Then Bayes's theorem (and every other statement about probability) is true only when applied to statements about how often a certain event will occur. But of course, most people do assign probabilities to hypotheses and fixed parameters, even if only informally. Bayesian proba…

I think it's strange this sudden comeback of a theory that was dismissed more than 70 years ago by Fisher and many others, but no one, as far as I know, cares to explain why Fisher was wrong and why the theory is right. It makes me very suspicious, to be honest.

Re: Bayes's Theorem: What's the Big Deal?

#73

Post by sharkbot » Tue, Jan 05, 2016, 6:05 PM UTC

IanCal wrote: post #31

Laaw wrote: post #14

I've been saying this for years , and this is a large reason why I find the LessWrong folks to be almost entirely full of it. Their inability to come up with accurate priors is completely lost on many of the folks who follow this kind of thinking. A couple of comments are saying, "no duh" to this article, but those folks likely don't realize quite how many other people are falling into this trap. "Garbage in, garbage…

Do priors just start you off closer to the truth? That is to say, if you start with any prior, will enough additional pieces of evidence always let you converge on the truth? Does anyone commonly set their priors to be a distribution? Perhaps a range or actually a normal distribution to represent a prior with uncertainty?

This comes to mind: https://en.wikipedia.org/wiki/Aumann's_agreement_theorem

Essentially, two genuine Bayesian rationalists (with some hand wavy preconditions) cannot agree to disagree; ie, they will eventually converge onto the same understanding of an event.

Re: Bayes's Theorem: What's the Big Deal?

#74

Post by astine » Tue, Jan 05, 2016, 6:05 PM UTC

TeMPOraL wrote: post #47

astine wrote: post #37

Earlier quoted context omitted.

"Is using mathy concepts to dress up poor reasoning worse than not using anything to back up your reasoning?" I believe so. If your belief is baseless, or based on flimsy evidence or simple bias, it's best if that's obvious. Dressing up weak reasoning to seem stronger is a form of lying. It's what we call sophistry. A big part of the problem is that for a lot of people don't understand the math well enough to point o…

Honesty is an ultimate issue here. If my reasoning is shoddy, but I plug it into some math apparatus, then it'll likely make my problems obviously wrong. If my reasoning is very inaccurate and the data uncertain, being precise about it can at least make the results salvageable. Scott Alexander argues for this position quite well in [0]. Humans can lie with statistics well. But they can lie with plain language even be…

"If my reasoning is shoddy, but I plug it into some math apparatus, then it'll likely make my problems obviously wrong."

That's pretty clearly untrue. I remember reading a study recently where the p value was less than .01 or something like that but where the experimental design was clearly flawed. The correlation wasn't the correlation they thought they had. But because the math looked good and it was easier than actually reviewing the experiment, it was tempting to take the study on face value.

I've read Scott's essay before and I understand his argument, but I don't think it works. While, you might be able to avoid some bad reasoning simply by being more systematic, you can also strengthen bad arguments with a faulty application of statistics. What Scott doesn't do is provide an analysis of how often each of these things happens. I'd argue that for each time a quick application of statistics save someone from a bad intuitive judgment, a misapplication of statistics is used to encourage a bad judgment at least one time if not more.

Understand that my argument here is not that one should never use statistics or even Bayes theorem, but that a naive or lazy application can be worse than no application.

Re: Bayes's Theorem: What's the Big Deal?

#75

Post by sageikosa » Tue, Jan 05, 2016, 6:07 PM UTC

theseatoms wrote: post #3

> The potential for Bayes abuse begins with P(B), your initial estimate of the probability of your belief, often called the “prior.” tldr; priors matter

This was Dawkins' assessment in the use of Bayesian Probabilities to "prove" the god hypothesis...basically the numbers being entered were complete and utter fabrications, making the mathematics pointless except that they lent an air of quantitative rigor.

Re: Bayes's Theorem: What's the Big Deal?

#76

Post by gipp » Tue, Jan 05, 2016, 6:08 PM UTC

bmh100 wrote: post #70

skybrian wrote: post #56

Earlier quoted context omitted.

How does that work? It doesn't sound like a well-defined distribution since the area under the curve needs to be 1.

It has to do with calculus. If the probability of a given result approaches zero as the result itself approaches positive or negative infinity, then the area under the curve approaches 1. Imagine 1/2 + 1/4 + 1/8 ... to infinity. The sum approaches 1 as the denominator approaches infinity. With calculus, we can determine that with mathematical methods.

I think he was referring specifically to an unbounded uniform distribution, which is indeed not well-defined.

Re: Bayes's Theorem: What's the Big Deal?

#77

Post by logfromblammo » Tue, Jan 05, 2016, 6:09 PM UTC

They were doing great until they got up to the re-test, claiming that the second positive gives you 99% certainty you have cancer. That only works if the second test is completely independent from the first. If you repeat the first test a second time, only for those who get a positive result on the first, the same condition that caused a false positive on the first can cause a false positive on the second.

In reality, a cheap blood or urine test is likely to be followed by a more expensive test on a second portion of the same sample, then by an even more expensive tissue biopsy. Redoing an identical test only reduces random errors. It does not address the diagnostic bias of the test itself.

For instance, a pregnancy test detects hCG, from the placenta of a developing fetus. A man with various types of cancer or liver disease may also produce hCG, and can therefore produce a false positive for every test of that type, no matter how many repetitions. This does not give him greater confidence that he is pregnant!

Understanding Bayes also requires an understanding of event independence! For truly independent events, P(x|y) = P(x) and P(y|x) = P(y).

Re: Bayes's Theorem: What's the Big Deal?

#78

Post by RodericDay » Tue, Jan 05, 2016, 6:14 PM UTC

TeMPOraL wrote: post #62

RodericDay wrote: post #57

Earlier quoted context omitted.

I know you're a huge advocate for Lesswrong, but not "everybody" or "anyone" has quotes ripe for picking like Yudkowsky. Stephen Bond is not just throwing some opinion out there, he's backing it up with first-hand sources: Yudkowsky on his simplified views of why race gets brought up: > "Race adds extra controversy to everything; in that sense, it's obvious what difference skin colour makes politically". > "Group inj…

I'm not that huge advocate of LW. I used to frequent the site, I read the Sequences and generally liked what I saw. RE parts you quoted. Of course they look bad, because they've been ripped out from context. Race: http://lesswrong.com/lw/kk/why_are_individual_iq_differences... This is an article asking, to quote: "But why is it that the rest of the world seems to think that individual genetic differences are okay, wh…

I guess I just disagree with you that they're taken out of context?

1. Bond posits Yudkowski thinks that racism is mostly due to genetic difference, and not about the deliberate, mostly political disenfranchisement of minorities.

You say this is a wrong because his appraisal of the situation as being caused by genetic disparities is benevolent, and that he wants to work to "fix" it.

These aren't in contradiction- to say that it's unfortunate that society is racialized, but that it's due to traits rather than politics, is better than being wantonly racist, but still a well-known form of fallacious racism. Some would say this latter one is actually worse in terms of perpetuating the situation.

2. Of course it's "controversial from the get go". He looks at the disproportionate representation of certain people in ie: tech, and concludes that it's because they're "more competent". Nothing here seems to be a counter, you're just basically saying "it's politically incorrect so it's probably right".

3. Clearly it's just a story. This point eludes exactly nobody. The whole point is that these kind of people find these questions ("is rape really bad?") really intriguing rather than obvious. It makes you wonder about the power of Bayesian reasoning- the exact point of the essay.

I think you're too uncharitable to critical writers like Bond (it took me a while to acquire a taste for this vicious kind of writing), underestimating their and their audiences' understanding. As a result, you think this context adds way more than it does.

Re: Bayes's Theorem: What's the Big Deal?

#79

Post by jasonwatkinspdx » Tue, Jan 05, 2016, 6:15 PM UTC

IanCal wrote: post #31

Laaw wrote: post #14

I've been saying this for years , and this is a large reason why I find the LessWrong folks to be almost entirely full of it. Their inability to come up with accurate priors is completely lost on many of the folks who follow this kind of thinking. A couple of comments are saying, "no duh" to this article, but those folks likely don't realize quite how many other people are falling into this trap. "Garbage in, garbage…

Do priors just start you off closer to the truth? That is to say, if you start with any prior, will enough additional pieces of evidence always let you converge on the truth? Does anyone commonly set their priors to be a distribution? Perhaps a range or actually a normal distribution to represent a prior with uncertainty?

> if you start with any prior, will enough additional pieces of evidence always let you converge on the truth?

That's the general trend: pooling data tends to make priors converge. However, converging priors isn't quite the same thing as everyone converging on the truth.

Statistical expressions themselves are an incomplete explanation, we routinely use assumptions about the direction of causality that aren't captured in them. See Judea Pearl's "Why I am only half Bayesian" paper for a discussion as well as an intro to how his framework approaches such independence assumptions.

Re: Bayes's Theorem: What's the Big Deal?

#80

Post by jedberg » Tue, Jan 05, 2016, 6:15 PM UTC

> Cognitive scientists conjecture that our brains incorporate Bayesian algorithms as they perceive, deliberate, decide.

As a Cognitive scientist myself, this amused me. The reason being because in the 1950s, Cognitive scientists thought that the brain worked like telephone switching equipment.

Basically, we fit our current model of cognition to the most popular model of computing at the time. Looks like the trend hasn't stopped (although to be fair we we were talking about the Baysian model of cognition 20 years ago, so at least that one lasted a while).