Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bayes's Theorem: What's the Big Deal?

blogs.scientificamerican.com

Post reply on HN
Print view View original

101–110 of 267 posts

Re: Bayes's Theorem: What's the Big Deal?

#101

Post by ucha »

darawk wrote: post #89

> If you get tested again, you can reduce your uncertainty I've always been bothered by statements like this about medical tests. This assumes that false positives are statistically independent. But isn't it more likely in general that false positives would be highly correlated in individuals, test administrators, or labs? E.g. If the same person takes the same test from the same doctor and sends it to the same lab,…

That's a nitpick on a correct statement. Unless two tests are always perfectly correlated, you will reduce your uncertainty. They don't need to be independent.

Re: Bayes's Theorem: What's the Big Deal?

#102

Post by jsprogrammer »

RogerL wrote: post #8

I prefer to think of it in terms of the statistical inversion problem. That is, we have an event(s) that occur, which we may imperfectly understand. We take noisy measurements of that event. Clearly, the causal relationship is the events cause the measurements - a bad measurement does not cause the event to move. But, in practice all we have are measurements, and from that we want to find an optimal (or good) estimat…

Most measurements do affect the event.

Re: Bayes's Theorem: What's the Big Deal?

#103

Post by jedberg »

darawk wrote: post #89

> If you get tested again, you can reduce your uncertainty I've always been bothered by statements like this about medical tests. This assumes that false positives are statistically independent. But isn't it more likely in general that false positives would be highly correlated in individuals, test administrators, or labs? E.g. If the same person takes the same test from the same doctor and sends it to the same lab,…

You could also look at it the other way: Using the same doctor and lab and procedure is the best way to eliminate a false positive, because if the cause was external, then the cause may not be repeated. But if you went to a new lab/doctor/whatever, you've now introduced new variables that could cause a false positive on top of whatever already caused it.

Given that it could go either way, it makes sense to think of each one as independent.

Now if you really wanted to take advantage of Bayes, if you got a positive test then you should get two more tests, one with the same lab and one with a totally independent lab (or even if you got a negative test, assuming your first Bayes run gives a 50% confidence)

Re: Bayes's Theorem: What's the Big Deal?

#104

Post by lordCarbonFiber »

sago wrote: post #46
lordCarbonFiber wrote: post #7

Earlier quoted context omitted.

That was my thought as well. Garbage in = garbage out, that's pretty standard in most fields. I really didn't like how the author treated the theorem as if it's some sort of magic, aside from something everyone that's taken a college prob/stat class has derived from first principles.

The problem is that advocates do treat it as a magical thing. They extrapolate from the fact it is proven to the claim that all knowledge is Bayesian, to the implication that all Bayesian reasoning is knowledge. This fashion is why, for example, BT has been used to both prove the resurrection of Jesus, and to prove that Jesus didn't exist: both to a very high probability.

I'm almost sad I've never met one of these people in the wild. I'd really like to sit down and watch someone, with a straight face, try to say they both have a set probability for their belief on Jesus AND the probability that some vague ~evidence~ would exist given that belief.

Re: Bayes's Theorem: What's the Big Deal?

#105

Post by TeMPOraL »

astine wrote: post #74
TeMPOraL wrote: post #47

Earlier quoted context omitted.

Honesty is an ultimate issue here. If my reasoning is shoddy, but I plug it into some math apparatus, then it'll likely make my problems obviously wrong. If my reasoning is very inaccurate and the data uncertain, being precise about it can at least make the results salvageable. Scott Alexander argues for this position quite well in [0]. Humans can lie with statistics well. But they can lie with plain language even be…

"If my reasoning is shoddy, but I plug it into some math apparatus, then it'll likely make my problems obviously wrong." That's pretty clearly untrue. I remember reading a study recently where the p value was less than .01 or something like that but where the experimental design was clearly flawed. The correlation wasn't the correlation they thought they had. But because the math looked good and it was easier than ac…

I see your point and I agree.

For myself, I try to limit myself to the mathematical apparatus I feel comfortable with. I know that if I were to open a statistics textbook, I could find something to plug in my estimates and reach a conclusion, and I'm pretty sure the conclusion would be bullshit. I learned it the hard way in high school - I remember the poor results of trying to solve math and physics homework assignments on topics I didn't understand yet. The mistakes were often subtle, but devastating.

Re: Bayes's Theorem: What's the Big Deal?

#106

Post by jsprogrammer »

mikeash wrote: post #24
njohnson41 wrote: post #2

Good article. I'm only a bit disappointed that the author seems not to realize that Bayes' theorem is just a simple consequence of probability theory, and should be attractive not because "maybe the brain is Bayesian", but because it is based on sound set-theoretic and analytic principles. If Bayes' theorem is false, so is probability theory, and so is nearly everything we know about probability. Edit: Here is a good…

Just because a theorem is true doesn't mean you can't misuse it or that you don't need to do some work to map it to reality. For example, the Banach-Tarski theorem is solid, but that doesn't mean you can start a business making golf balls by buying one and then endlessly replicating it.

Certainly. Just because you can name a theorem doesn't mean you can derive it either. The article had no actual computation or derivation of the theorem. Instead, it talked about beliefs and other things that don't really exist (in regards to the computation of a probability value).

Re: Bayes's Theorem: What's the Big Deal?

#107

Post by bordercases »

Chathamization wrote: post #95
TeMPOraL wrote: post #66

Earlier quoted context omitted.

You can't protect yourself in 100% - it would require developing more powerful reasoning tools in an infinite regression. But what you can do is to use introspection, and triple-check your reasoning when it seems to defy common sense or leads you to weird (awful) conclusions. That's why LW is so big on biases and heuristics by the way - you can treat them as a list of warning signs; if your reasoning seems to match s…

The problem with trying to rely on heuristics to avoid biases is people often ignore the biases in the heuristics of choice. To continue the example of LW, there are many people there who seem to think highly of IQ test, and who ignore the many issues with them (the Flynn effect an the effect of incentives being a couple examples of the flaws in IQ tests). Trying to remove biases is great. But there is a problem when…

You should probably know what James Flynn thinks of the Flynn effect. He doesn't try to escape the conclusions of an I.Q. test as much as extend them, despite some of the "paradoxes".

Re: Bayes's Theorem: What's the Big Deal?

#108

Post by TeMPOraL »

Chathamization wrote: post #95
TeMPOraL wrote: post #66

Earlier quoted context omitted.

You can't protect yourself in 100% - it would require developing more powerful reasoning tools in an infinite regression. But what you can do is to use introspection, and triple-check your reasoning when it seems to defy common sense or leads you to weird (awful) conclusions. That's why LW is so big on biases and heuristics by the way - you can treat them as a list of warning signs; if your reasoning seems to match s…

The problem with trying to rely on heuristics to avoid biases is people often ignore the biases in the heuristics of choice. To continue the example of LW, there are many people there who seem to think highly of IQ test, and who ignore the many issues with them (the Flynn effect an the effect of incentives being a couple examples of the flaws in IQ tests). Trying to remove biases is great. But there is a problem when…

Yes, there is that risk, and no doubt many fall for it. Ego / self-esteem issues may be a big part of it. But then again, every worthy goal poses risks. When you fly a plane, there's a greater risk you'll kill yourself than when you stay on the ground, and yet airplanes are being flown and we're reaping great benefits from it.

RE IQ, personally, I'm 100% confused on the topic. I used to believe that Flynn effect is basically people getting better at doing tests, but recently I heard that someone controlled for that and the effect remained. So I don't know. The topic is complicated and most of studies I heard of are the kind of psychology and social science I implicitly assume is mostly bullshit.

Re: Bayes's Theorem: What's the Big Deal?

#109

Post by jacobolus »

woodchuck64 wrote: post #60
Laaw wrote: post #14

I've been saying this for years , and this is a large reason why I find the LessWrong folks to be almost entirely full of it. Their inability to come up with accurate priors is completely lost on many of the folks who follow this kind of thinking. A couple of comments are saying, "no duh" to this article, but those folks likely don't realize quite how many other people are falling into this trap. "Garbage in, garbage…

> I've been saying this for years, and this is a large reason why I find the LessWrong folks to be almost entirely full of it. Their inability to come up with accurate priors is completely lost on many of the folks who follow this kind of thinking. I assume you're saying LessWrong folks are more prone to miscalculating priors than most. Could you give some examples of this?

The LessWrong folks aren’t obviously better or worse at calculating priors than anyone else. The “problem” is that their hobby is spending their free time considering outlandish scenarios, inventing arbitrary assumptions related to such scenarios, drawing questionable conclusions, and then convincing themselves that because they used logic and math, their analysis must be correct. Plenty of other folks who spend time on similar activities with a less pseudo-rigorous framing end up as conspiracy theorists or occultists; belief in AI overlords ruling humanity, the technological singularity, cryogenics, or impending 1000-year human lifespans is far from the kookiest thing people convince themselves about.

Re: Bayes's Theorem: What's the Big Deal?

#110

Post by Retric »

XFrequentist wrote: post #35
Laaw wrote: post #14

I've been saying this for years , and this is a large reason why I find the LessWrong folks to be almost entirely full of it. Their inability to come up with accurate priors is completely lost on many of the folks who follow this kind of thinking. A couple of comments are saying, "no duh" to this article, but those folks likely don't realize quite how many other people are falling into this trap. "Garbage in, garbage…

FWIW, it seems to me that a major benefit of the Bayesian approach is to make bad reasoning (in the form of, say, an unreasonable prior) transparent and obvious. I've never heard it claimed that the Bayesian approach was robust to sophisticated idiocy (neither on LessWrong nor mainstream writing on Bayesian methods), except in the narrow techical sense that the posterior asymptotically approximates the likelihood giv…

Bayesian seems great when you first see it. It should be obvious how to apply it for something like a card game. The problem is how long would it take you to realize a deck of cards was missing the 4 of diamonds? What if the card was lost 1/2 way though the game? How about on the prior hand?

In the end it's stuck at one level of recursion and all facts are fuzzy.

Post reply on HN