What's the difference between a prior and a bias?
How does one distinguish between the two?
The Bayesian Cringe (2021)
11–20 of 27 posts
Re: The Bayesian Cringe (2021)
#12What's the difference between a prior and a bias? How does one distinguish between the two?
A prior is whatever you start with. There’s literally no requirements. Bayes tells you how to update your priors, whatever they are, in the face of new data. Nothing more, nothing less. In principle it doesn’t matter what priors you start with (how biased they are), in the sense that given enough data, your likelihoods will converge to what is really the case.
Re: The Bayesian Cringe (2021)
#13I noticed a shift in my attitude in strong priors when I switched from academia to industry and have only recently realized why. When doing an analysis in an academic setting, the goal is to get a paper past reviewers to be published. And the reviewers were adversaries that were trying to disprove your work (at best these were helpful critique; at worst they were bad-faith nit-pickers that were looking for any excuse…
It’s definitely a learned soft skill to direct interactions to be collaborative rather than adversarial.
It’s too easy to fall into an adversarial discussion because of differing opinions (eg about code architecture) when really you’re on the same team. I try to keep in mind (and convey) the image of “you and me side to side against the problem on the whiteboard” rather than “you and me against each other”
Re: The Bayesian Cringe (2021)
#14What's the difference between a prior and a bias? How does one distinguish between the two?
A prior is whatever you start with. There’s literally no requirements. Bayes tells you how to update your priors, whatever they are, in the face of new data. Nothing more, nothing less. In principle it doesn’t matter what priors you start with (how biased they are), in the sense that given enough data, your likelihoods will converge to what is really the case.
If you have the right model.
Re: The Bayesian Cringe (2021)
#15I still have no clue what Gelman is saying about anything ever, and this post is no exeception. He seems like a great guy in interviews and presentations but anything he writes or talks about is highly non-specific.
I find that it's rather specific and niche, and maybe that's why you aren't getting it.
Re: The Bayesian Cringe (2021)
#16I noticed a shift in my attitude in strong priors when I switched from academia to industry and have only recently realized why. When doing an analysis in an academic setting, the goal is to get a paper past reviewers to be published. And the reviewers were adversaries that were trying to disprove your work (at best these were helpful critique; at worst they were bad-faith nit-pickers that were looking for any excuse…
Really interesting insight, yo add, in academia you are trying to prove yourself to the reviewers, wheras in business settings, you are offering services. Different goals.
Re: The Bayesian Cringe (2021)
#17There's also the fact that a prior is really hard to explain to someone else. By definition, it's the unexplainable starting point! Yet when I lay out fairly tight Bayesian reasoning, there's always that one person sucking life out of the entire conversation with "Wait can you go back to that first number? How did you arrive at that?" and it's an unanswerable question because any attempt would have to start from anot…
I hoped we were past the point where people describe priors as "subjective". Sigh. Bayes formula can be applied with complete objectivity. Inference given the same starting knowledge plus the same evidence will always yield the same conclusion. It need not be subjective, but it IS relative to one's knowledge. And why is that a bad thing? Shouldn't more knowledge == better inference? We have a systematic way to build knowledge with reasonable objectivity (the scientific method) - should we not use it? Or do these people literally believe that less knowledge somehow improves their decision making?
Re: The Bayesian Cringe (2021)
#18There's also the fact that a prior is really hard to explain to someone else. By definition, it's the unexplainable starting point! Yet when I lay out fairly tight Bayesian reasoning, there's always that one person sucking life out of the entire conversation with "Wait can you go back to that first number? How did you arrive at that?" and it's an unanswerable question because any attempt would have to start from anot…
I hoped we were past the point where people describe priors as "subjective". Sigh. Bayes formula can be applied with complete objectivity. Inference given the same starting knowledge plus the same evidence will always yield the same conclusion. It need not be subjective, but it IS relative to one's knowledge. And why is that a bad thing? Shouldn't more knowledge == better inference? We have a systematic way to build…
It seems that there’s still unavoidable subjectivity in making the choice of prior distribution? I get how it’s objective for a fixed choice, but my understanding is that you need to first make that choice in order to be objective. Is it actually that making your choice of prior is obvious (or there is some objectively optimal way to pick a prior), which rules out any subjectivity in the choice of prior?
Re: The Bayesian Cringe (2021)
#19Earlier quoted context omitted.
I hoped we were past the point where people describe priors as "subjective". Sigh. Bayes formula can be applied with complete objectivity. Inference given the same starting knowledge plus the same evidence will always yield the same conclusion. It need not be subjective, but it IS relative to one's knowledge. And why is that a bad thing? Shouldn't more knowledge == better inference? We have a systematic way to build…
It seems that there’s still unavoidable subjectivity in making the choice of prior distribution? I get how it’s objective for a fixed choice, but my understanding is that you need to first make that choice in order to be objective. Is it actually that making your choice of prior is obvious (or there is some objectively optimal way to pick a prior), which rules out any subjectivity in the choice of prior?
Yes. You've hit the essence of the problem - the choice of a prior. The scientific method gives us a way to choose a prior that fits reality. You are claiming something different, that our prior is based on subjective choices. Yep, those are both ways to choose a prior. But only one is guaranteed to be valid. Do I even need to mention the appalling history of base rate neglect in medical research? Neglecting priors leads to bad decisions.
Re: The Bayesian Cringe (2021)
#20Earlier quoted context omitted.
I hoped we were past the point where people describe priors as "subjective". Sigh. Bayes formula can be applied with complete objectivity. Inference given the same starting knowledge plus the same evidence will always yield the same conclusion. It need not be subjective, but it IS relative to one's knowledge. And why is that a bad thing? Shouldn't more knowledge == better inference? We have a systematic way to build…
It seems that there’s still unavoidable subjectivity in making the choice of prior distribution? I get how it’s objective for a fixed choice, but my understanding is that you need to first make that choice in order to be objective. Is it actually that making your choice of prior is obvious (or there is some objectively optimal way to pick a prior), which rules out any subjectivity in the choice of prior?
There is also unavoidable subjectivity in making the choice of model, data, etc.