Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bayes's Theorem: What's the Big Deal?

blogs.scientificamerican.com

Post reply on HN
Print view View original

91–100 of 267 posts

Re: Bayes's Theorem: What's the Big Deal?

#91

Post by snowwrestler »

darawk wrote: post #89

> If you get tested again, you can reduce your uncertainty I've always been bothered by statements like this about medical tests. This assumes that false positives are statistically independent. But isn't it more likely in general that false positives would be highly correlated in individuals, test administrators, or labs? E.g. If the same person takes the same test from the same doctor and sends it to the same lab,…

These are questions that themselves would need to be answered with scientific research. In the absence of real empirical data, I'd be uncomfortable with saying it is "more likely in general that false positives would be highly correlated." Without data, we don't know how likely it is.

Re: Bayes's Theorem: What's the Big Deal?

#92

Post by haberman »

I am only just learning about this stuff, but there are several things in this article that seem incorrectly explained. Conceptual clarity is paramount to me, so it drove me a little crazy!

> Bayes’ theorem is a method for calculating the validity of beliefs (hypotheses, claims, propositions) based on the best available evidence (observations, data, information).

Bayes theorem is a statement about probability, not "validity." This description makes it sound like Bayes theorem is a function BT(belief, evidence) = validity of belief. But it's not like that at all.

Probability is a way of measuring uncertainty. Things are uncertain for two main reasons: either we can't observe them directly ("do I have this disease or not?") or they haven't happened yet ("what side will this coin flip land on?"). (If you believe in a deterministic universe, the second is just a special case of the first.)

The "beliefs" (aka priors/posteriors) in Bayes theorem are statements of probability. To use the article's example, if it is claimed that 1% of the population has a certain disease, your "belief" or "prior" is that P(I have the disease) = 0.01. The article seems to get confused and think that the "belief" here is "I have the disease." Bayes theorem doesn't tell you about "the probability that a belief is true" like the article says, the belief is a probability. It also doesn't tell you if your belief is "valid."

Bayes theorem takes your existing belief about the probability of something and gives you a new probability that incorporates some evidence you observed.

Re: Bayes's Theorem: What's the Big Deal?

#93

Post by jsprogrammer »

Fomite wrote: post #42
IanCal wrote: post #31

Earlier quoted context omitted.

Do priors just start you off closer to the truth? That is to say, if you start with any prior, will enough additional pieces of evidence always let you converge on the truth? Does anyone commonly set their priors to be a distribution? Perhaps a range or actually a normal distribution to represent a prior with uncertainty?

In my field (Epidemiology), when doing Bayesian analysis, it is very common to set one's priors to be a distribution. Sometimes the point estimate and spread of a previously conducted study or meta-analysis, sometimes merely a uniform distribution with upper and lower bounds ("It is extremely unlikely that the relative risk of disease for this exposure is below 0.01 or above 100...") It's been argued that frequentist…

Bayesian analysis comes directly from the probability axioms, which are 'frequentist'.

Re: Bayes's Theorem: What's the Big Deal?

#94

Post by TeMPOraL »

Jabbles wrote: post #67
Anderkent wrote: post #50

Earlier quoted context omitted.

Doctors observe a result of the test, and know the basic probabilities (in the example, 99% test accuracy, 1% of population have the disease). The problem is that they [often] draw incorrect conclusions from those observations (99% test accuracy and you tested positive? well then you likely - 99% - have the disease, right?). The question formed as 'your one patient tested positively' is more immediately relevant, I'd…

Of course, doctors do not randomly assign tests to patients. Their prior that a patient has a disease is a lot higher than the background frequency of it occurring. Getting them to estimate their prior would be interesting.

It depends. Some tests are applied without prior suspicion, so you deal with exactly the background frequency. With others, the disease in question is so rare that false positives will dominate even if the doctor has serious suspicions. The second case is the reason for the "think horses not zebras" aphorism.

Doing a few explicit Bayesian calculations can help one internalize just how much important is that often forgotten P(A) factor is.

The same applies, by the way, to the antiterrorism security theater - many support it just because they have no intuition (or idea) about base rates.

Re: Bayes's Theorem: What's the Big Deal?

#95

Post by Chathamization »

TeMPOraL wrote: post #66
zepto wrote: post #59

Earlier quoted context omitted.

How do you be extra careful except by developing yet more powerful reasoning tools?

You can't protect yourself in 100% - it would require developing more powerful reasoning tools in an infinite regression. But what you can do is to use introspection, and triple-check your reasoning when it seems to defy common sense or leads you to weird (awful) conclusions. That's why LW is so big on biases and heuristics by the way - you can treat them as a list of warning signs; if your reasoning seems to match s…

The problem with trying to rely on heuristics to avoid biases is people often ignore the biases in the heuristics of choice. To continue the example of LW, there are many people there who seem to think highly of IQ test, and who ignore the many issues with them (the Flynn effect an the effect of incentives being a couple examples of the flaws in IQ tests).

Trying to remove biases is great. But there is a problem when someone works to remove some biases, then believes that they are inherently more rational than the public at large, and then uncritically accepts there other biases ("Someone like me who's worked hard to remove there biases must be correct when compared to the biased masses.").

Re: Bayes's Theorem: What's the Big Deal?

#96

Post by mhb »

snowwrestler wrote: post #91
darawk wrote: post #89

> If you get tested again, you can reduce your uncertainty I've always been bothered by statements like this about medical tests. This assumes that false positives are statistically independent. But isn't it more likely in general that false positives would be highly correlated in individuals, test administrators, or labs? E.g. If the same person takes the same test from the same doctor and sends it to the same lab,…

These are questions that themselves would need to be answered with scientific research. In the absence of real empirical data, I'd be uncomfortable with saying it is "more likely in general that false positives would be highly correlated." Without data, we don't know how likely it is.

Which is why the author of the article needs to mention that the second test is independent of the first.

Re: Bayes's Theorem: What's the Big Deal?

#97

Post by PaulHoule »

3pt14159 wrote: post #33
PaulHoule wrote: post #16

So can frequentism. Many investigators in parapsychology who were sincere and intelligent appear to have based their career on the incorrect use of frequentist statistics. And it's not just them. Ernerst Rutherford, who discovered the atomic nucleus, "If your experiment needs statistics, you ought to do a better experiment." In the 1990s I was a physics grad student and I think none of the professors had ever heard o…

I'm a little confused. Are you saying that a tenure track prof wrote a paper on how to evauluate fitted power law curves? Was it something else besides least squares? Because I can't possibly see this getting accepted to a statistics journal.

Remember what a parameter estimator is.

If you sample 100 values out of a very large pool and add them up then divide by 100 what you get is not the mean of the distribution, but an estimator of the mean of the distribution as you would get a slightly different answer if you picked a different 100.

Often estimators are simple formulas like that (they are for power laws) but there are subtle details, for instance to get the standard deviation in that case you might think you divide by N (100) but you really should divide by N-1 (99).

Back in the 1990s myself and the people around knew some popular statistic formulas but not the concept of estimating a parameter.

And of course its not physics. Social scientists and life science people tend to take a course on statistics but it is fair to say that the median paper in those fields has some mistake in how they do statistics.

Re: Bayes's Theorem: What's the Big Deal?

#99

Post by sophacles »

darawk wrote: post #89

> If you get tested again, you can reduce your uncertainty I've always been bothered by statements like this about medical tests. This assumes that false positives are statistically independent. But isn't it more likely in general that false positives would be highly correlated in individuals, test administrators, or labs? E.g. If the same person takes the same test from the same doctor and sends it to the same lab,…

I think this is a good question - I hope someone who actually knows about this stuff chimes in with some answers.

In the mean time as a thought experiment, going with a blood test example:

* levels of the compound being tested may rise or fall naturally, and the test checks for a certain concentration or above. * the patient may have the condition, but not have had it long enough for markers to have risen to "trigger" levels. * The patient may not exactly follow pre-draw instructions on eating etc, skewing results * in the case of false negative: the person's immune response may be temporarily suppressing the marker * in the case of something like cell counts - this sample could just be a random local variation * The tech or doctor or whoever could randomly make a mistake on one sample, but not on each sample.

And so on. My devil's advocate point here is that the patient, doctor, lab, and so on are not deterministic code - there are a lot of random inputs in the entire process chain.

Re: Bayes's Theorem: What's the Big Deal?

#100

Post by aamar »

hyperpape wrote: post #20

Interesting that they mention the medical case, when there's some psychological work around the idea that we should present these cases in terms of natural frequencies instead of Bayes' theorem. The natural frequencies approach is to say "if 10000 people take the test, 100 will have cancer. Of them, 99 will get an accurate positive test, and 1 will have a false negative test. Of the other 9900, 99 will receive a fals…

It seems likely that this will help some doctors in at least simple cases, but natural frequencies don't 'compose' well if you are doing multiple tests; you can't use them to compute posterior probabilities when combining a half-dozen different tests, at least not without involving impractically huge numbers or fractional people.

Moreover, the trend in modern medicine is towards combining binary/categorical tests with a range of distributions based on age, gender, race, genetic factors, vitals, continuous lab result values, etc. Yes, we can theoretically delegate some or all of that to software, but that is true for medical diagnosis in general; while we're not there, doctors must understand and perform these calculations.

So it seems fine to use natural frequencies to help out, but building Bayesian intuitions early and often seems a better path.

Post reply on HN