Live data from Hacker News

Interview on ”Bayesian Statistics the Fun Way”

notamonadtutorial.com

21–30 of 54 posts

Re: Interview on ”Bayesian Statistics the Fun Way”

#21
post #12

Earlier quoted context omitted.

Bayesian Data Analysis by Andrew Gelman http://www.stat.columbia.edu/~gelman/book/

one vote for BDA. For programmers who learn better by implementing things, this book [1] is also good: [1]: https://www.amazon.com/Bayesian-Methods-Hackers-Probabilisti...

Parts of that book are available online[1] for free. If not for that book I would never have understood how to apply Bayesian stats to problems that interested me.

[1] http://camdavidsonpilon.github.io/Probabilistic-Programming-...

Re: Interview on ”Bayesian Statistics the Fun Way”

#22
post #7

As someone who uses statistics all the time at work, I sympathize so much with this article and greatly enjoyed it. Every time I try to introduce a Bayesian prior, coworkers either look at me like I'm crazy (because they've never heard of or used Bayesian stats) or like I've suddenly gone soft and introduced a bunch of nebulous, touchy-feely context into the objective truth (if they're dedicated frequentists). Then w…

> like I've suddenly gone soft and introduced a bunch of nebulous, touchy-feely context into the objective truth This drives me nuts. If you haven't, check out the paper "Beyond subjective and objective in statistics" by Gelman and Hennig (2017). Right at the beginning they make the point that any analysis includes external information in many ways, such as adjusting variables for imbalance, how we deal with outliers…

This critique might come from the idea that having a good analytic model, or at least some valuable analytic insights, involves much more than assigning some priors. Of course, the two things don't exclude each other, but for some frequentists Bayesians have the wrong perspective - or at least that's the critique, whether it's true or not.

Another issue that I personally have with Bayesianism is that I believe that assigning probabilities to singular events is only meaningful and admissible at all if there is a good analytic explanation for the respective propensity. For example, we may be able to deduce that a die is reasonably fair from the way it is constructed and our knowledge of physics, and later confirm this by frequentist analysis. Merely believing or claiming that the die is fair is not acceptable. Again, the difference is only one of attitude in the end, I suppose.

Maybe philosophers have given Bayesian statistics a bad rap, too, because many of those who call themselves Bayesians are also "probabilists", i.e., they think that rational belief must conform to the probability calculus. There are many arguments against probabilism and the only arguments that speak for it are Dutch book arguments. The view does not have very strong foundations.

Re: Interview on ”Bayesian Statistics the Fun Way”

#23
Are there conditions in which a Bayesian reasoner is obliged to make racist decisions? Say you're a Bayesian infant who has never met anyone but Mom, who has blue skin, and has always been very nice to you. You then meet 100 other people of various colors, 97 of who are also nice to you. The other 3 people were mean to you, and they all have purple skin, and only they have purple skin.

Person 101 approaches you, and they have purple skin. As a Bayesian baby you start to cry, anticipating meanness.

According to the idea that discrimination by immutable characteristics is bad, and that we should judge people as individuals, this Bayesian choice to cry is a symptom of a racist mindset. Isn't it?

In what circumstances should Bayesian priors _not_ be used for decision making, in deference to wider principles of justice?

Re: Interview on ”Bayesian Statistics the Fun Way”

#24
post #23

Are there conditions in which a Bayesian reasoner is obliged to make racist decisions? Say you're a Bayesian infant who has never met anyone but Mom, who has blue skin, and has always been very nice to you. You then meet 100 other people of various colors, 97 of who are also nice to you. The other 3 people were mean to you, and they all have purple skin, and only they have purple skin. Person 101 approaches you, and…

This seems like pretty flawed reasoning. What you are describing is not a prior but a posterior - the distribution after the observed data has been taken into account.

If anything the prior can help make you less racist by incorporating the knowledge that immutable characteristics are not good indicators of danger/not danger.

The thing is though, even if you are told race doesn't matter through a prior, if you observe a strong correlation over many instances it's going to be hard to ignore that regardless of your prior (what you are told). While it may not be a causal relationship, it may still be a good predictor.

Re: Interview on ”Bayesian Statistics the Fun Way”

#25
post #23

Are there conditions in which a Bayesian reasoner is obliged to make racist decisions? Say you're a Bayesian infant who has never met anyone but Mom, who has blue skin, and has always been very nice to you. You then meet 100 other people of various colors, 97 of who are also nice to you. The other 3 people were mean to you, and they all have purple skin, and only they have purple skin. Person 101 approaches you, and…

A purpose of statistics is to make generalizations about a population. The solution to your problem is to have a larger sample size.

Re: Interview on ”Bayesian Statistics the Fun Way”

#27
post #8

Can anyone recommend a 'Bayesian statistics the hard way' book?

For the hard way, look at Bruno de Finetti's Theory of Probability:

https://onlinelibrary.wiley.com/doi/book/10.1002/97811192863...

Jaynes is certainly very deep and some sections are harder than others. It's interesting regardless of your level (this is a book worth rereading several times).

For a less technical, but full of insight, introduction see Dennis Lindley's Understanding Uncertainty:

https://onlinelibrary.wiley.com/doi/book/10.1002/97811186501...

Re: Interview on ”Bayesian Statistics the Fun Way”

#28

Earlier quoted context omitted.

> like I've suddenly gone soft and introduced a bunch of nebulous, touchy-feely context into the objective truth This drives me nuts. If you haven't, check out the paper "Beyond subjective and objective in statistics" by Gelman and Hennig (2017). Right at the beginning they make the point that any analysis includes external information in many ways, such as adjusting variables for imbalance, how we deal with outliers…

This critique might come from the idea that having a good analytic model, or at least some valuable analytic insights, involves much more than assigning some priors. Of course, the two things don't exclude each other, but for some frequentists Bayesians have the wrong perspective - or at least that's the critique, whether it's true or not. Another issue that I personally have with Bayesianism is that I believe that a…

My understanding of physics is that no die toss can be considered "fair" because such a macroscopic system behaves deterministically according to Newton's laws, and isn't even too chaotic to model accurately. No matter the shape or balance of the die, the outcome is determined by the initial conditions and the toss. A skilled gambler can make a fair die land however they want.

The only thing I know is that a well-made die is symmetrical, and so if I have no prior knowledge of its initial orientation then I have to use a uniform prior because nothing else has the requisite symmetry group.

The same could be said for a die that is just sitting on the table without having been observed by me yet, no toss needed.

Re: Interview on ”Bayesian Statistics the Fun Way”

#29
post #19
post #10

As someone who has a master's degree in statistics and often uses Bayesian statistics, I think we should not focus on whether one is a Bayesian or a frequentist, but rather be pragmatic and take the most practical approach to solving a statistical problem. Moreover, I think statistical education should start with frequentist concepts and then extend them to the Bayesian framework since the likelihood plays also a maj…

Respectfully, I find that people who are not statisticians overwhelming disagree with your point on which should be taught first. Bayesian is a natural order of inference for people. The whole concept of the black swan ("all swans are white") proves this out. Frequentist statistics is much less intuitive to people. My preference is for people to be able to use some statistics, and Bayesian gets them productive faster…

Frequentist statistics is often pretty poorly taught. Ideas like likelihood, modeling, and optimization underly the mechanics of both worlds. There's a big obsession with testing, but the Neyman Pearson testing framework is sound an intuitive.

Bayesian statistics gets a big boost because it's usually taught as a system instead of as a recipe book.

Post reply on HN