Bayes's Theorem: What's the Big Deal? • Hacker News Board

Re: Bayes's Theorem: What's the Big Deal?

#81

Post by astine » Tue, Jan 05, 2016, 6:16 PM UTC

TeMPOraL wrote: post #36

astine wrote: post #10

Earlier quoted context omitted.

I think you're missing the broader argument, which is using 'mathy' concepts to dress up poor reasoning. Obviously priors matter, but what matters most of all is how good/complete your evidence is. Using a mathematical formula to lend credence to weak evidence (through liberal use of assumptions) is a hallmark of pseudoscience. The same could be said of many of the abuses of statistics and Bayes theorem is merely one…

This is a general argument against statistics . Or math, in general. Yes, dressing your bullshit in math can make people believe you more, but it doesn't change the fact that you're lying. Are we supposed to stop using math for good because evil people are using it for evil?

No, it's an argument against using statistics without first considering the strength of your data.

Re: Bayes's Theorem: What's the Big Deal?

#82

Post by NN88 » Tue, Jan 05, 2016, 6:17 PM UTC

TeMPOraL wrote: post #38

XFrequentist wrote: post #35

Earlier quoted context omitted.

FWIW, it seems to me that a major benefit of the Bayesian approach is to make bad reasoning (in the form of, say, an unreasonable prior) transparent and obvious. I've never heard it claimed that the Bayesian approach was robust to sophisticated idiocy (neither on LessWrong nor mainstream writing on Bayesian methods), except in the narrow techical sense that the posterior asymptotically approximates the likelihood giv…

Yup. Actually, a common theme on LessWrong was realizing that with better reasoning tools you're more able to bullshit yourself , and so you need to be extra-careful.

"The first principle is that you must not fool yourself and you are the easiest person to fool."

- Richard P. Feynman

Re: Bayes's Theorem: What's the Big Deal?

#83

Post by hyperpape » Tue, Jan 05, 2016, 6:23 PM UTC

vinchuco wrote: post #51

hyperpape wrote: post #20

Interesting that they mention the medical case, when there's some psychological work around the idea that we should present these cases in terms of natural frequencies instead of Bayes' theorem. The natural frequencies approach is to say "if 10000 people take the test, 100 will have cancer. Of them, 99 will get an accurate positive test, and 1 will have a false negative test. Of the other 9900, 99 will receive a fals…

Except when https://en.m.wikipedia.org/wiki/Simpson%27s_paradox

It's a lovely phenomenon, but what are you trying to say?

Re: Bayes's Theorem: What's the Big Deal?

#84

Post by 1024core » Tue, Jan 05, 2016, 6:24 PM UTC

XFrequentist wrote: post #35

Laaw wrote: post #14

I've been saying this for years , and this is a large reason why I find the LessWrong folks to be almost entirely full of it. Their inability to come up with accurate priors is completely lost on many of the folks who follow this kind of thinking. A couple of comments are saying, "no duh" to this article, but those folks likely don't realize quite how many other people are falling into this trap. "Garbage in, garbage…

FWIW, it seems to me that a major benefit of the Bayesian approach is to make bad reasoning (in the form of, say, an unreasonable prior) transparent and obvious. I've never heard it claimed that the Bayesian approach was robust to sophisticated idiocy (neither on LessWrong nor mainstream writing on Bayesian methods), except in the narrow techical sense that the posterior asymptotically approximates the likelihood giv…

> I've never heard it claimed that the Bayesian approach was robust to sophisticated idiocy

Alas, nothing is robust to sophisticated idiocy.

Re: Bayes's Theorem: What's the Big Deal?

#85

Post by philh » Tue, Jan 05, 2016, 6:25 PM UTC

I feel like there are multiple questions getting conflated:

* Does Bayes help honest enquirers to find the truth? (Relative to what? At what skill level?)

* Does Bayes help bullshitters to hide the truth? (Relative to what? At what skill level of bullshitter and mark?)

Re the second, I think it's difficult to use Bayes to bullshit someone who understands Bayes as well as you do.

Re: Bayes's Theorem: What's the Big Deal?

#86

Post by hyperpape » Tue, Jan 05, 2016, 6:27 PM UTC

Anderkent wrote: post #50

hyperpape wrote: post #44

Earlier quoted context omitted.

I'm not quite sure what you're saying. Doctors don't observe probabilities or enormous frequencies. Either way, there are good odds that this is information that someone is communicating to them, not the result of their personal experience.

Doctors observe a result of the test, and know the basic probabilities (in the example, 99% test accuracy, 1% of population have the disease). The problem is that they [often] draw incorrect conclusions from those observations (99% test accuracy and you tested positive? well then you likely - 99% - have the disease, right?). The question formed as 'your one patient tested positively' is more immediately relevant, I'd…

I certainly agree that thinking in terms of percentages is more familiar. But it's familiarity that doesn't help most people get the right answer.

Re: Bayes's Theorem: What's the Big Deal?

#87

Post by crpatino » Tue, Jan 05, 2016, 6:27 PM UTC

Anderkent wrote: post #18

astine wrote: post #10

Earlier quoted context omitted.

I think you're missing the broader argument, which is using 'mathy' concepts to dress up poor reasoning. Obviously priors matter, but what matters most of all is how good/complete your evidence is. Using a mathematical formula to lend credence to weak evidence (through liberal use of assumptions) is a hallmark of pseudoscience. The same could be said of many of the abuses of statistics and Bayes theorem is merely one…

Is using mathy concepts to dress up poor reasoning worse than not using anything to back up your reasoning? At least you can point out exactly what's wrong with the mathy reasoning. A colleague of mine says 'Sometimes pulling numbers out of your arse and using them to make a decision is better than pulling a decision out of your arse'

> 'Sometimes pulling numbers out of your arse and using them to make a decision is better than pulling a decision out of your arse'

Agreed! Leaving the pseudoscience example aside - since there are strong emotions involved - we can clearly see that it is indeed useful and necessary to take decisions under uncertain/incomplete information. This is advantageous whenever the cost of inaction is expected to exceed the cost of backtracking a less than perfect decision, which often is the case.

Let's say... project management. IF you take the time to find out that your project requires 100 tasks, 30 of which lay in your critical path; you can argue if each task will take one day or one week to complete, and you can debate whether adding a 3rd or 4th member to the team will significantly speed up the completion date or not. But you will definitevely be in better shape than if your PM just cook up some 5-page-spec overnight and commited to have it running in beta test by the end of the month before even anouncing it to the team...

Which itself will be better than having all your potential contracts snatched by competitors that never do any estimation at all but are very good at pulling themselves out of tarpits of their own making.

Re: Bayes's Theorem: What's the Big Deal?

#88

Post by hyperpape » Tue, Jan 05, 2016, 6:28 PM UTC

Jabbles wrote: post #67

Anderkent wrote: post #50

Earlier quoted context omitted.

Doctors observe a result of the test, and know the basic probabilities (in the example, 99% test accuracy, 1% of population have the disease). The problem is that they [often] draw incorrect conclusions from those observations (99% test accuracy and you tested positive? well then you likely - 99% - have the disease, right?). The question formed as 'your one patient tested positively' is more immediately relevant, I'd…

Of course, doctors do not randomly assign tests to patients. Their prior that a patient has a disease is a lot higher than the background frequency of it occurring. Getting them to estimate their prior would be interesting.

Note that the frequency with which people who don't have the condition take the test is a major (but not the only) component of a good prior.

Re: Bayes's Theorem: What's the Big Deal?

#89

Post by darawk » Tue, Jan 05, 2016, 6:34 PM UTC

> If you get tested again, you can reduce your uncertainty

I've always been bothered by statements like this about medical tests. This assumes that false positives are statistically independent. But isn't it more likely in general that false positives would be highly correlated in individuals, test administrators, or labs? E.g. If the same person takes the same test from the same doctor and sends it to the same lab, it seems extremely unlikely that the results will be independent. And to me at least, it seems highly likely that a false positive will correlate with some aspect of the individual's biological (e.g. some similar substance in the blood to what's being tested), and as such even using a different doctor/lab would not be all that likely to ameliorate this issue.

Re: Bayes's Theorem: What's the Big Deal?

#90

Post by lqdc13 » Tue, Jan 05, 2016, 6:34 PM UTC

I just love how some guy, who by his own admission read a little Wikipedia on the topic, is critiquing a statistical method.

It would make for a much stronger argument if he actually showed some numbers where people are getting the priors wrong. That is, how often people get the priors wrong and the probability of mistake if they used a different strategy more commonly used in the field.