Live data from Hacker News

Bayesian statistics for confused data scientists

nchagnet.pages.dev

51–60 of 65 posts

Re: Bayesian statistics for confused data scientists

#51

[flagged]

The exact opposite is true. Virtually everyone’s intuition is aligned with the Bayesian model. That intuition has to be hammered out of people in their stats classes because for decades frequentist approaches were computationally more feasible, even if they don’t align with how most humans interpret probability.

[flagged]

Re: Bayesian statistics for confused data scientists

#52
post #25

Earlier quoted context omitted.

I feel like I'm a polyglot here but primarily a native frequentist thinker. I've found Bayesian methods shine in cases of an "intractible partition function". Cases such as language models, where the cardinality of your discrete probability distribution is extremely large, to the point of intractability. Bayesians tend to immediately go to things like Monte Carlo estimation. Is that fundamentally Bayesian and anti-fr…

When you are using something like Monte Carlo you’re probably using some method that’s more advanced than the Naïve Bayes, is that right?

I'm talking about, for something simple, the negative sampling in word2vec.

Or the temperature setting for an LLM etc.

Re: Bayesian statistics for confused data scientists

#53
post #30

Earlier quoted context omitted.

Huh? Are there really any pure frequentists post Stein's paradox? At least ones that are aware of it and maintain objections to fusing the fields?

> Are there really any pure frequentists post Stein's paradox? What does that have to do with anything? If one cares about that using a shrinkage estimator is an option which maintains the frequentist purity.

There is some frequentist procedure there, but it seems hard to not recognize the deep connection to Bayesian statistics and wonder if you should begin to question your baseline assumptions. Since the entire justification for using a shrinkage estimator has a whole lot more in common with the foundations of Bayesian statistics than it does with the foundations of frequentist stats.

Purist frequentists using a shrinkage estimator looks a lot like heliocentric Ptolemic astronomy.

Re: Bayesian statistics for confused data scientists

#54

Earlier quoted context omitted.

Huh? Are there really any pure frequentists post Stein's paradox? At least ones that are aware of it and maintain objections to fusing the fields?

Downvote me all you want. Bayesianism is misapplied much more frequently than frequentism. It just makes it way too easy to fudge p values. Sorry not sorry.

I do always laugh when I see a Bayesian object to p-values, then use a Bayesian procedure that is mathematically identical to treating p values as posterior probabilities.

Just saying the word "Bayesian" doesn't actually make it different

Re: Bayesian statistics for confused data scientists

#55
post #5

I went through grad school in a very frequentist environment. We “learned” Bayesian methods but we never used them much. In my professional life I’ve never personally worked on a problem that I felt wasn’t adequately approached with frequentist methods. I’m sure other people’s experiences are different depending on the problems you gravitate towards. In fact, I tend to get pretty frustrated with Bayesian approaches b…

> I’ve never personally worked on a problem that I felt wasn’t adequately approached with frequentist methods Multilevel models are one example of problem were Bayesian methods are hard to avoid as otherwise inference is unstable, particularly when available observations are not abundant. Multilevel models should be used more often as shrinking of effect sizes is important to make robust estimates. Lots of flashy res…

Curious what you might consider “adequate shrinking”?

Horshoe priors, partial pooling, something more?

I realize that might be highly subject

Re: Bayesian statistics for confused data scientists

#56
post #5

Earlier quoted context omitted.

> I’ve never personally worked on a problem that I felt wasn’t adequately approached with frequentist methods Multilevel models are one example of problem were Bayesian methods are hard to avoid as otherwise inference is unstable, particularly when available observations are not abundant. Multilevel models should be used more often as shrinking of effect sizes is important to make robust estimates. Lots of flashy res…

Curious what you might consider “adequate shrinking”? Horshoe priors, partial pooling, something more? I realize that might be highly subject

I guess this depends on the problem at hand.

But I was thinking about a typical hierarchical model with partial pooling and standard weakly informative priors.

Re: Bayesian statistics for confused data scientists

#57

Earlier quoted context omitted.

Downvote me all you want. Bayesianism is misapplied much more frequently than frequentism. It just makes it way too easy to fudge p values. Sorry not sorry.

I do always laugh when I see a Bayesian object to p-values, then use a Bayesian procedure that is mathematically identical to treating p values as posterior probabilities. Just saying the word "Bayesian" doesn't actually make it different

It’s mathematically identical but conceptually different. The things that go into the calculation are different, the numbers that get out of the calculation mean different things. Laughing is healthy though.

Re: Bayesian statistics for confused data scientists

#58
post #57

Earlier quoted context omitted.

I do always laugh when I see a Bayesian object to p-values, then use a Bayesian procedure that is mathematically identical to treating p values as posterior probabilities. Just saying the word "Bayesian" doesn't actually make it different

It’s mathematically identical but conceptually different. The things that go into the calculation are different, the numbers that get out of the calculation mean different things. Laughing is healthy though.

You can just shortcut all of that if you're a Bayesian and just plain say "p-values are posterior probabilities under a uniform (improper) prior" and save everyone a lot of time.

And if you're doing that, don't care complain that p-values can be misinterpreted, because you're basically just laundering the misinterpretation of p-values.

Sure, you are mathematically pure because you made an initial assumption that it can be so, rather than being confused, but the end result is the same.

Re: Bayesian statistics for confused data scientists

#59

I went through grad school in a very frequentist environment. We “learned” Bayesian methods but we never used them much. In my professional life I’ve never personally worked on a problem that I felt wasn’t adequately approached with frequentist methods. I’m sure other people’s experiences are different depending on the problems you gravitate towards. In fact, I tend to get pretty frustrated with Bayesian approaches b…

Bayesian methods are not better than frequentist methods and vice versa. I use both, but mostly Bayesian.

Bayesian approaches take a long time thinking, making models, choosing priors, simulations etc. but they provide a better estimate and understanding the parameters. I hate point estimates and decision based arbitrary p-value. Whenever possible I use Bayesian methods.

Re: Bayesian statistics for confused data scientists

#60
What a stupid idea to put a pesky GIF in the middle of enough complicated article. Isn't the freaking article supposed to be readable? Play your stupid meme once than go south. It even optimized to look good in a loop-form, what a stupid mad world if some math article causes a trouble like this.
Post reply on HN