Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bayesian Data Analysis, Third edition (2013) [pdf]

sites.stat.columbia.edu

Post reply on HN
Print view View original

51–60 of 71 posts

Re: Bayesian Data Analysis, Third edition (2013) [pdf]

#51

Post by musebox35 »

pyyxbkshed wrote: post #20
kianN wrote: post #13

This is my favorite book on statistics. Full stop. The author Andrew Gelman created a whole new branch of Bayesian statistics with both his theoretical work on hierarchical modeling while also publishing Stan to enable practical applications of hierarchical models. It took me about a year to work through this book on the side (including the exercises) and it provided the foundation for years of fruitful research into…

What is a book / course on statistics that I can go through before this so that I can understand this?

I found the book from David Mackay on Information Theory, Inference, and Learning Algorithms to be well written and easy to follow. Plus it is freely available from his website: https://www.inference.org.uk/itprnn/book.pdf

It goes through fundamentals of Bayesian ideas in the context of applications in communication and machine learning problems. I find his explanations uncluttered.

Re: Bayesian Data Analysis, Third edition (2013) [pdf]

#52

Post by some_guy_nobel »

g9yuayon wrote: post #30

I can attest how useful Bayesian analysis is. My team recently needed to sample from many millions of items to test their qualities. The question is that given a certain budget and expectation, what's the minimum or maximum number of items that we need to sample. There was an elegant solution to this problem. What was surprising, though, was how reluctant the engineers are to learn such basic techniques. It's not lik…

What were they reluctant to learn? Why do they need to learn it?

Plenty of engineers have to take an introductory stats course, but it's not clear why you'd want your engineers to learn bayesian statistics? I would be surprised if they could correctly interpret a p-value or regression coefficient, let alone one with interaction effects. (It'd be wholly useless if they could, fwiw).

It'd be nice if the statisticians/'data scientists' on my team learned their way around the CI/CD pipelines, understood kubernetes pods, and could write their own distributed training versions of their pytorch models, but division-of-labor is a thing for a reason, and I don't expect them to nor need them to.

Re: Bayesian Data Analysis, Third edition (2013) [pdf]

#53

Post by some_guy_nobel »

I first found Gelman from this wonderful visual explanation of Linear Regression:

https://mlu-explain.github.io/linear-regression/

It cited Regression and Other Stories (though not the Bayesian chapters, which I'm now inspired to dig into before checking this out).

Re: Bayesian Data Analysis, Third edition (2013) [pdf]

#54

Post by djmips »

kianN wrote: post #13

This is my favorite book on statistics. Full stop. The author Andrew Gelman created a whole new branch of Bayesian statistics with both his theoretical work on hierarchical modeling while also publishing Stan to enable practical applications of hierarchical models. It took me about a year to work through this book on the side (including the exercises) and it provided the foundation for years of fruitful research into…

Can you explain to me in simple terms how your fruitful research benefited you in a concrete way. Is this simply an enlightening hobby or do you have significant everyday applications? What kind of cool job has you employ Bayesian Data Analysis day to day and for what benefit? How do the suits relate to such knowledge and it's beneficial application that may be well beyond their ken?

Re: Bayesian Data Analysis, Third edition (2013) [pdf]

#56

Post by memming »

analog31 wrote: post #39
j7ake wrote: post #31

Is Bayesian data analysis relevant anymore in the era of foundation models and big data?

General quantitative thinking, and a sense of statistics, are still valuable. If you don't learn them from Bayes specifically, you should learn them somehow. The "square root of n rule" is still a stern master. And we're still not past having to think about whether our results make sense. [0] The rule of thumb that signal-to-noise improves with the square root of the number of measurements. Also, as my dad put it: "T…

Your dad sounds like an awesome person.

Re: Bayesian Data Analysis, Third edition (2013) [pdf]

#57

Post by jmpeax »

pyyxbkshed wrote: post #20
kianN wrote: post #13

This is my favorite book on statistics. Full stop. The author Andrew Gelman created a whole new branch of Bayesian statistics with both his theoretical work on hierarchical modeling while also publishing Stan to enable practical applications of hierarchical models. It took me about a year to work through this book on the side (including the exercises) and it provided the foundation for years of fruitful research into…

What is a book / course on statistics that I can go through before this so that I can understand this?

Statistical Rethinking by Richard McElreath. He even has a youtube series covering the book if you prefer that modality.

Re: Bayesian Data Analysis, Third edition (2013) [pdf]

#58

Post by piqufoh »

While we're here - I've gained a lot from "Data Analysis: A Bayesian Tutorial" by DS Sivia and J Skilling. It's a graduate level text, and I found the chapters very concise and the subject well-laid out. It was one of those books that gave me continuous insight and fresh inspiration - even though it's more than 10 years old.

Re: Bayesian Data Analysis, Third edition (2013) [pdf]

#59

Post by jmpeax »

j7ake wrote: post #31

Is Bayesian data analysis relevant anymore in the era of foundation models and big data?

Yes, the two are orthogonal concepts. Text did not disappear just because we invented photography. Bayesian data analysis is for inverse problems, such as using data to learn about the properties of the system/model that could have generated the data, and neural networks are for forward problems such as using data to generate more data or make predictions.

You can use BDA for forward problems too, via posterior predictive samples. The benefit over neural networks for this task is that with BDA you get dependable uncertainty quantification about your predictions. The disadvantage is that the modalities are somewhat limited to simple structured data.

You can also use neural networks for inverse problems, such as for example with Neural Posterior Estimation. This approach shows promise since it can tackle more complex problems than the standard BDA approach of Markov Chain Monte Carlo and with much faster results, but the accuracy and dependability are still quite lacking.

Re: Bayesian Data Analysis, Third edition (2013) [pdf]

#60

Post by CrazyStat »

kianN wrote: post #50
nextos wrote: post #47

Earlier quoted context omitted.

He co-wrote the reference textbook on the topic and made interesting methodological contributions, but Gelman acknowledges other people as creators of the theoretical underpinnings of multilevel/hierarchical modeling, including Stein or Donoho [1]. The field is quite old, one can find hierarchical models in articles that were published many decades ago. Also, IMHO, his best work has been done describing how to do sta…

Totally agree and great point that hierarchical models have been around for a long time; however, these were primarily analytical, leveraging conjugate priors or requiring pretty extensive integration. I would say his work with Stan and his writings, along with theorists like Radford Neal, really opened the door to a computational approach to hierarchical modeling. And I think this is a meaningfully different field.

I give Gelman a lot of credit for popularizing hierarchical models, but you give him too much.

Before Stan existed we used BUGS [1] and then JAGS [2]. And most of the work on computation (by Neal and others) was entirely independent of Gelman.

[1] https://en.wikipedia.org/wiki/Bayesian_inference_using_Gibbs...

[2] https://en.wikipedia.org/wiki/Just_another_Gibbs_sampler

Post reply on HN