Bayes's Theorem: What's the Big Deal? • Hacker News Board

Re: Bayes's Theorem: What's the Big Deal?

#161

Post by hyperpape » Tue, Jan 05, 2016, 10:05 PM UTC

marcosdumay wrote: post #149

hyperpape wrote: post #83

Earlier quoted context omitted.

It's a lovely phenomenon, but what are you trying to say?

Maybe that the natural frequency "normatized" for some people may be completely different from the actual frequency that apply to them, because the people that researched the frequency did their grouping badly.

I'm not sure if that's relevant. It seems like either you know the actual frequency for their subgroup, in which case you redo the numbers in the example, or you don't in which case you can't accommodate that regardless of how you calculate.

A natural frequency is different way of expressing and reasoning about the same information as percentages.

Re: Bayes's Theorem: What's the Big Deal?

#162

Post by Chathamization » Tue, Jan 05, 2016, 10:10 PM UTC

cobaltblue wrote: post #147

Chathamization wrote: post #95

Earlier quoted context omitted.

The problem with trying to rely on heuristics to avoid biases is people often ignore the biases in the heuristics of choice. To continue the example of LW, there are many people there who seem to think highly of IQ test, and who ignore the many issues with them (the Flynn effect an the effect of incentives being a couple examples of the flaws in IQ tests). Trying to remove biases is great. But there is a problem when…

IQ is held in high esteem because research around g is very good and comprehensive, perhaps the crown jewel of psychology. Additionally most people's knowledge of the Flynn effect is out of date -- recent studies (here's one: http://www.sciencedirect.com/science/article/pii/S0160289615... in fact here's a boatload of references http://www.iapsych.com/iqmr/fe/MasterFlynnEffectreferencelis... ) show a rise, leveling of…

When fairly minor monetary incentives (over $10) can lead to a 20 point increase in scores[1], I'd be wary of reading too much into the tests. As for the Flynn effect, I've read a number of different theories, but there doesn't seem to be any consensus. Given the other issues IQ tests have (like the one just mentioned), I'd be wary about assuming that it doesn't stem from underlying problems with the test itself. Some researchers seem to think it stems from familiarity with test taking in general, which seems to match the general understanding that you will do better on IQ tests if you repeatedly take them, referred to as the "practice effect" (IE, IQ tests at least in part measure familiarity with the test).

[1] http://news.sciencemag.org/2011/04/what-does-iq-really-measu...

Re: Bayes's Theorem: What's the Big Deal?

#163

Post by darawk » Tue, Jan 05, 2016, 10:14 PM UTC

ucha wrote: post #101

darawk wrote: post #89

> If you get tested again, you can reduce your uncertainty I've always been bothered by statements like this about medical tests. This assumes that false positives are statistically independent. But isn't it more likely in general that false positives would be highly correlated in individuals, test administrators, or labs? E.g. If the same person takes the same test from the same doctor and sends it to the same lab,…

That's a nitpick on a correct statement. Unless two tests are always perfectly correlated , you will reduce your uncertainty. They don't need to be independent.

Sorry, I should have provided a more complete quote:

> If you get tested again, you can reduce your uncertainty enormously, because your probability of having cancer, P(B), is now 50 percent rather than one percent. If your second test also comes up positive, Bayes’ theorem tells you that your probability of having cancer is now 99 percent, or .99.

This statement is incorrect if the results are correlated at all.

Re: Bayes's Theorem: What's the Big Deal?

#164

Post by Chathamization » Tue, Jan 05, 2016, 10:20 PM UTC

bordercases wrote: post #107

Chathamization wrote: post #95

Earlier quoted context omitted.

The problem with trying to rely on heuristics to avoid biases is people often ignore the biases in the heuristics of choice. To continue the example of LW, there are many people there who seem to think highly of IQ test, and who ignore the many issues with them (the Flynn effect an the effect of incentives being a couple examples of the flaws in IQ tests). Trying to remove biases is great. But there is a problem when…

You should probably know what James Flynn thinks of the Flynn effect. He doesn't try to escape the conclusions of an I.Q. test as much as extend them, despite some of the "paradoxes".

Fair enough. It's also worth pointing out that Alfred Binet, generally considered to be one of the fathers (if not THE father) of intelligence testing, felt that the idea of quantitative intelligence testing was severely flawed (and had fairly harsh words for people who believed that there was largely a single measure of intelligence).

Re: Bayes's Theorem: What's the Big Deal?

#165

Post by n4r9 » Tue, Jan 05, 2016, 10:20 PM UTC

tzs wrote: post #112

Referring to an appearance of Bayes' theorem on Sheldon's whiteboard: > Bayes’ theorem has become so popular that it even made a guest appearance on the hit CBS show Big Bang Theory That's not a sign that it has become popular, any more than the appearance of the standard human pedigree notation used by genetic counselors on the same whiteboard indicates that standard human pedigree notation has become popular among…

P(popular | appeared on BBT) < 1

Re: Bayes's Theorem: What's the Big Deal?

#166

Post by cobaltblue » Tue, Jan 05, 2016, 10:33 PM UTC

Chathamization wrote: post #162

cobaltblue wrote: post #147

Earlier quoted context omitted.

IQ is held in high esteem because research around g is very good and comprehensive, perhaps the crown jewel of psychology. Additionally most people's knowledge of the Flynn effect is out of date -- recent studies (here's one: http://www.sciencedirect.com/science/article/pii/S0160289615... in fact here's a boatload of references http://www.iapsych.com/iqmr/fe/MasterFlynnEffectreferencelis... ) show a rise, leveling of…

When fairly minor monetary incentives (over $10) can lead to a 20 point increase in scores[1], I'd be wary of reading too much into the tests. As for the Flynn effect, I've read a number of different theories, but there doesn't seem to be any consensus. Given the other issues IQ tests have (like the one just mentioned), I'd be wary about assuming that it doesn't stem from underlying problems with the test itself. Som…

This study isn't surprising, I'm unsure what you think it implies. Indeed IQ researchers have been wary themselves and known about motivational effects for decades, along with many other objections to testing like cultural bias and the like that in good modern research are all accounted for -- obviously if I sleep during a test I'll score very low, but if I'm actually awake and care I can increase my score by a huge factor. IQ isn't a perfect correlate, but it's common to reason as if it alone has a predictive power of 0.4-0.6 for various important things, that is stronger than any other single factor we know about. When you add in Conscientiousness (the big five traits being the other contender for psychology's crown jewel, I think), which is about grit, intrinsic motivation, and the like, together with IQ (two factors now, not just one), you get predictive powers of 0.7 to educational success. I would wager that the differences seen in the paper cited by your article are almost entirely accounted for by Conscientiousness, but from the abstract (don't have the full paper) it looks like that was not controlled for at all.

For your pleasure: https://ideas.repec.org/p/nbr/nberwo/15898.html

Re: Bayes's Theorem: What's the Big Deal?

#167

Post by auvrw » Tue, Jan 05, 2016, 10:33 PM UTC

modal logic is an equally valid way to get at a lot of the quandaries associated with bayesian ideas.

bayes' thm by itself is totally not a "big deal." the idea that every probability has an associated prior, even if it's not explicitly written in the notation (so think "prior of prior") is an interesting attempt to cope with uncertainty in a rigorous way.

i do agree that in some places in the sciences, while "the numbers don't lie," the stats can be misleading. still, i can understand why it's useful to make some statements with statistics in order to quickly arrive at some first-order approximations.

Re: Bayes's Theorem: What's the Big Deal?

#168

Post by ac29 » Tue, Jan 05, 2016, 10:39 PM UTC

edanm wrote: post #157

jacobolus wrote: post #109

Earlier quoted context omitted.

The LessWrong folks aren’t obviously better or worse at calculating priors than anyone else. The “problem” is that their hobby is spending their free time considering outlandish scenarios, inventing arbitrary assumptions related to such scenarios, drawing questionable conclusions, and then convincing themselves that because they used logic and math, their analysis must be correct. Plenty of other folks who spend time…

So I'm a LessWronger and know a bit about the "movement", and think you are misunderstanding what "LessWrongers think". Obviously not all LessWrongers think the same thing at all, but I'm talking about the average position of the people who believe AI safety should be worked towards. I'd love to explain the basic position, and tell me where you disagree with it. This is the basic position: 1. Intelligence can be crea…

> 1. Intelligence can be created, because there is nothing "special"/"magical" about humans, and our intelligence was eventually created.

Human intelligence evolved through a (very long!) series of natural processes, to the best of my knowledge. To say it was "created" implies something closer to a religious or philosophical opinion, rather than something supported by science.

> 2. At some point, humanity will create an "artificial general intelligence". (Since we'll just keep improving science and technology, and there's no fundamental reason why this won't eventually allow us to create an intelligence.

This is hugely debatable. Why is AGI inevitable? Even given great amounts of computing resources, a artificial general intelligence does not just automatically appear, it must somehow be designed and programmed. Fields like computer vision have grown tremendously using techniques like deep learning, but there really isn't any evidence that I know of that a general intelligence is any closer than it was 20 years ago.

Re: Bayes's Theorem: What's the Big Deal?

#169

Post by sorokod » Tue, Jan 05, 2016, 10:42 PM UTC

njohnson41 wrote: post #2

Good article. I'm only a bit disappointed that the author seems not to realize that Bayes' theorem is just a simple consequence of probability theory, and should be attractive not because "maybe the brain is Bayesian", but because it is based on sound set-theoretic and analytic principles. If Bayes' theorem is false, so is probability theory, and so is nearly everything we know about probability. Edit: Here is a good…

Given that

  p(A and B) = p(B and A)
  p(A and B ) = p(A) * p(B|A)

we have:

  p(A) * p(B|A) = p(B) * p(A|B)
  p(B|A) = p(B) * p(A|B) / p(A)

Re: Bayes's Theorem: What's the Big Deal?

#170

Post by Avshalom » Tue, Jan 05, 2016, 10:49 PM UTC

edanm wrote: post #157

jacobolus wrote: post #109

Earlier quoted context omitted.

The LessWrong folks aren’t obviously better or worse at calculating priors than anyone else. The “problem” is that their hobby is spending their free time considering outlandish scenarios, inventing arbitrary assumptions related to such scenarios, drawing questionable conclusions, and then convincing themselves that because they used logic and math, their analysis must be correct. Plenty of other folks who spend time…

So I'm a LessWronger and know a bit about the "movement", and think you are misunderstanding what "LessWrongers think". Obviously not all LessWrongers think the same thing at all, but I'm talking about the average position of the people who believe AI safety should be worked towards. I'd love to explain the basic position, and tell me where you disagree with it. This is the basic position: 1. Intelligence can be crea…

Well "is able to e.g. cure cancer" is not actually very general. Which leads to the problem with 2) whats the economics behind creating a general intelligence when a specific intelligence will get you better results in a given industry. Even then specific intelligence is still going to be subject to the good-enough economic plateau that has killed so many future predictions.

Then the problems with 4 on up really concern the speed with which 4 can feasibly happen. The AI goes FOOM doomsaysers seem to think that we'll end up with an AI which is so horribly inefficient that/and it will be able to rewrite it self to be super duper intelligent without leaving its machine (and won't accidentally nerf itself in the attempt) and then that super duper intelligent computer will trick several industries into building an even more powerful body for itself etc... all of this happening before humans pull the plug. no step of which is has anything beyond speculation to support it.

In a general note the full employment theorems mean that even if general AI is economically incentivized there's still going to be dozens/hundred/thousands of different AIs carving out niches for themselves which, given that the earth/universe has limited resources, handily prevents the paper clip maximizer problem. While the future may not need humans it will still be a diverse future.