Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Bayes’ Rule Emerged Triumphant from Two Centuries of Controversy

mcgrayne.com

Post reply on HN
Print view View original

51–60 of 83 posts

Re: How Bayes’ Rule Emerged Triumphant from Two Centuries of Controversy

#51

Post by eli_gottlieb »

jordigh wrote: post #24

The biggest historical obstacle to Bayesian stats was low amounts of data available and difficulty of computation. Frequentist stats is optimised around these. With 30 samples and very easy computations, you are able to produce a reasonable frequentist confidence interval, sometimes even with less data. On the other hand, even the simplest Bayesian analysis of determining the probability of heads in a coin flip requi…

>Now that we have lots of data and lots of computing power, Bayesian stats can show its results, after getting rebranded as "machine learning".

Partly lots of computing power, and partly the Monte Carlo revolution that enabled us to replace intractable integrals with lots of cheap CPU time.

Re: How Bayes’ Rule Emerged Triumphant from Two Centuries of Controversy

#52

Post by kareemsabri »

danbruc wrote: post #47
kareemsabri wrote: post #43

Earlier quoted context omitted.

> The question is what difference it makes going from a large but finite to an infinite number of tosses. Convergence is only guaranteed as N -> ∞. The difference between large but finite and infinite is.. well, infinite :) So that's a pretty significant difference. > Either is impossible to get only heads an infinite number of times, then I have a problem understanding why that is, or all heads is still a possible o…

Okay, assuming that is true, is there an intuitive way to understand that? And the fact that lim[n-> ∞] 0.5^n = 0 unfortunately won't do for me, that is true for any specific infinite sequence, even those containing 50/50 heads and tails. I think most of the sequences - hand-waving, most of an infinite set - are 50/50 heads and tails just because there are more possibilities - hand-waving again - to arrange 50/50 hea…

> Okay, assuming that is true, is there an intuitive way to understand that? And the fact that lim[n-> ∞] 0.5^n = 0 unfortunately won't do for me, that is true for any specific infinite sequence, even those containing 50/50 heads and tails.

That's true, but only if you pick ONE specific sequence. The likelihood of any precise sequence of heads/tails is equally unlikely as all heads. However, we are not fixing the outcomes ordering, but simply saying taken to infinity the distribution of outcomes will tend to 50/50.

> I still would not really understand what forces my coin to show tails eventually

Nothing forces it to show heads either.

> but if there were a measure showing that those two sets have measure 1 respectively 0 it would already be easier to swallow

Well, we've already proven it's impossible to have all heads infinitely. The same would be true of any non-uniform sequence. Any sequence that favours heads by any margin at all, taken to infinity, would necessarily contain an infinite sub-sequence that contains all heads, which we've proven is impossible by the limit to infinity.

Re: How Bayes’ Rule Emerged Triumphant from Two Centuries of Controversy

#53

Post by jerf »

danbruc wrote: post #47
kareemsabri wrote: post #43

Earlier quoted context omitted.

> The question is what difference it makes going from a large but finite to an infinite number of tosses. Convergence is only guaranteed as N -> ∞. The difference between large but finite and infinite is.. well, infinite :) So that's a pretty significant difference. > Either is impossible to get only heads an infinite number of times, then I have a problem understanding why that is, or all heads is still a possible o…

Okay, assuming that is true, is there an intuitive way to understand that? And the fact that lim[n-> ∞] 0.5^n = 0 unfortunately won't do for me, that is true for any specific infinite sequence, even those containing 50/50 heads and tails. I think most of the sequences - hand-waving, most of an infinite set - are 50/50 heads and tails just because there are more possibilities - hand-waving again - to arrange 50/50 hea…

"Okay, assuming that is true, is there an intuitive way to understand that?"

I mean this quite seriously and am not being sarcastic or dismissive: Probably not. I think the general concept of "intuitive" is that there is some experiential analog to the concept in the real world, with which we've had a lot of experience and can thus "intuit" the behavior. The real world does not include infinity.

You can develop mathematical intuitions, but I don't think that's what you were saying.

In the mathematical intuition sense, it's worth pointing out the probability of any given infinite series of coin flips is zero. The all-heads or all-tails series are not special that way. While this may not be the best way to intuit it, if a metaphysical you who would live forever sat down and started flipping coins, you will never at any point be done with flipping an infinite series of coins. You could sit there until you flip any finite series of coins, but you will never flip an infinite one, even with our unrealistic stipulations of life span. From that perspective, a probability of zero of flipping an infinite number of heads should seem reasonable; there is probability zero that you will ever have flipped an infinite number of coins.

Re: How Bayes’ Rule Emerged Triumphant from Two Centuries of Controversy

#54

Post by mvf0 »

danbruc wrote: post #34
daemonomania wrote: post #27

Earlier quoted context omitted.

If a coin were flipped arbitrarily many times, and it landed on heads each time, and if we had no other information about the coin , a frequentist would say that the probability the coin lands on heads is one. Now, you might protest and say, "But this is a fair coin that just happened to land on heads arbitrarily many times". But then you are relying on a prior definition of probability in order to justify your objec…

I am not sure if that really addresses my issue. My problem is that I don't see what eventually forces convergences to 50/50. We can use two or even better many coins or, at least superficially equivalent, one coin and let several experimenters take turns. The results will converge towards 50/50 but only with high probability, or at least I don't see why they alway would, i.e. why not at least one sequence could not…

Your problem is that you don't seem to have sufficient understanding of the concept of limits.

Re: How Bayes’ Rule Emerged Triumphant from Two Centuries of Controversy

#55

Post by dschiptsov »

dschiptsov wrote: post #3

Brexit is the best example so far. That painful dissonance between so called reality and these probabilistic models.

Probabilities makes sense only with absolutely certain things like a fair coin or a dice.

In cases where there is no absolute certainty about how many sides or dimensions your "dice" has and that it is not biased and that there is no other forces or factors in play probability ceases to make sense.

Probability of A, given B becomes meaningless when either A or B aren't precisely defined (like in the case of a "fair coin") and so is the relationship between the two.

Application of the Bayesian rule to "estimated" probabilities is just wrong and unscientific (in the face of ambiguity avoid the temptation to guess). Multiplying and dividing nonsense by nonsense yields nonsense.

The global financial crisis and recent cock-sure consensus about outcome of the brexit referendum the day before voting are good evidences.

Re: How Bayes’ Rule Emerged Triumphant from Two Centuries of Controversy

#56

Post by hacker42 »

arcanus wrote: post #2

Bayesian here. 'The theory that would not die' is a wonderful read, and notes how many scientists used Bayesian techniques (subjective probability) in a variety of contexts while the field was still unpopular (arguably, heretical) in the mainstream statistical community. Bayes was nevertheless used to inform ballistics calculations, help crack the enigma code, or inform search patterns for lost nuclear weapons. I don…

As a Bayesian you can maybe answer my question here: https://news.ycombinator.com/item?id=11985312

I would appreciate it!

Re: How Bayes’ Rule Emerged Triumphant from Two Centuries of Controversy

#57

Post by XFrequentist »

jordigh wrote: post #33
tgb wrote: post #29

Earlier quoted context omitted.

> On the other hand, even the simplest Bayesian analysis of determining the probability of heads in a coin flip requires some interesting integration and the somewhat obscure Beta distribution. Isn't this kind of misleading? The end result of the Beta distribution, etc. is just the extremely simple-to-compute Rule of Succession [1]. [1] https://en.wikipedia.org/wiki/Rule_of_succession

A bit, but in general, computing posterior distributions tends to very quickly lead to more complicated integrals. It just so happens that the Beta distribution is somewhat nice and symmetrical, if a bit obscure.

More generally, this is why there's (still) a mild obsession with "conjugate" prior/likelihood distribution pairs - i.e. combinations of prior and likelihood that give analytically tractable posteriors[1] - despite the ability to get easily get results with MCMC.

[1] Yes, everyone wants a nice posterior. You're very clever.

Re: How Bayes’ Rule Emerged Triumphant from Two Centuries of Controversy

#58

Post by [deleted] »

tingol wrote: post #36
amasad wrote: post #7

Earlier quoted context omitted.

As someone new to Bayesianism I'd be interested in hearing your experience applying it in day-to-day life. How useful do you think it is to the ordinary Joe? From my brief experience, after learning Bayes, my intuition about things involving probabilities grew very different than the people around me. For example, my friends were planning a skydiving trip and I googled the name of the skydiving business and found tha…

I think your friends were right. Peoples lives are not the same as a coin flip. You could almost surely say that whatever caused the accidents in the past has been a point of focus for the agency specifically so it never happens again. You can't really apply your statistical thinking in that scenario.

[deleted]

Re: How Bayes’ Rule Emerged Triumphant from Two Centuries of Controversy

#59

Post by amasad »

tingol wrote: post #36
amasad wrote: post #7

Earlier quoted context omitted.

As someone new to Bayesianism I'd be interested in hearing your experience applying it in day-to-day life. How useful do you think it is to the ordinary Joe? From my brief experience, after learning Bayes, my intuition about things involving probabilities grew very different than the people around me. For example, my friends were planning a skydiving trip and I googled the name of the skydiving business and found tha…

I think your friends were right. Peoples lives are not the same as a coin flip. You could almost surely say that whatever caused the accidents in the past has been a point of focus for the agency specifically so it never happens again. You can't really apply your statistical thinking in that scenario.

Maybe they were but they actually statistically came to that conclusion. In the same sense that people think that given that it has been all heads for the past 10 flips then the chance it is tails is higher in the next (it's not)

>You can't really apply your statistical thinking in that scenario.

The main idea behind Bayes is that you can apply probabilistic reasoning to anything where you don't have complete knowledge (which means, short of 1+1, is everything)

Re: How Bayes’ Rule Emerged Triumphant from Two Centuries of Controversy

#60

Post by amasad »

Jommi wrote: post #21
amasad wrote: post #7

Earlier quoted context omitted.

As someone new to Bayesianism I'd be interested in hearing your experience applying it in day-to-day life. How useful do you think it is to the ordinary Joe? From my brief experience, after learning Bayes, my intuition about things involving probabilities grew very different than the people around me. For example, my friends were planning a skydiving trip and I googled the name of the skydiving business and found tha…

Weren't your friends right? Since the incident happened, it's quite likely they fired an incompetnet instructor or tightened security protocols to such an effect that future dives would be safer.

Maybe but 3 years after the fatalities they had 2 serious injuries which makes me give less credence to this. (Also see my reply to sibling comment, my friends reasoned about this statistically while falling for the gamblers fallacy, sort of)
Post reply on HN