I think it would be interesting if frequentist stats can come up with more generative models. Current high level generative machine learning all rely on Bayesian modeling.
Bayesian statistics for confused data scientists
11–20 of 65 posts
Re: Bayesian statistics for confused data scientists
#12I think it would be interesting if frequentist stats can come up with more generative models. Current high level generative machine learning all rely on Bayesian modeling.
I'm not well versed enough, but what would a frequentist generative model even mean?
The entire generative concept implicitly assumes that parameters have probability distributions themselves that naturally give rise to generative models...
You could do frequentist inference on a generative model, sure, but generative modelling seems fundamentally alien to frequentist thinking?
Re: Bayesian statistics for confused data scientists
#13I think Rafael Irizarry put it best over a decade ago -- while historically there was a feud between self-declared "frequentists" and "Bayesians", people doing statistics in the modern era aren't interested in playing sides, but use a combination of techniques originating in both camps: https://simplystatistics.org/posts/2014-10-13-as-an-applied-...
That’s Bayesian propaganda
Huh? Are there really any pure frequentists post Stein's paradox? At least ones that are aware of it and maintain objections to fusing the fields?
Re: Bayesian statistics for confused data scientists
#14I went through grad school in a very frequentist environment. We “learned” Bayesian methods but we never used them much. In my professional life I’ve never personally worked on a problem that I felt wasn’t adequately approached with frequentist methods. I’m sure other people’s experiences are different depending on the problems you gravitate towards. In fact, I tend to get pretty frustrated with Bayesian approaches b…
> I’ve never personally worked on a problem that I felt wasn’t adequately approached with frequentist methods Multilevel models are one example of problem were Bayesian methods are hard to avoid as otherwise inference is unstable, particularly when available observations are not abundant. Multilevel models should be used more often as shrinking of effect sizes is important to make robust estimates. Lots of flashy res…
Thank you for Lindley's paradox! TIL
Re: Bayesian statistics for confused data scientists
#15I think it would be interesting if frequentist stats can come up with more generative models. Current high level generative machine learning all rely on Bayesian modeling.
I'm not well versed enough, but what would a frequentist generative model even mean? The entire generative concept implicitly assumes that parameters have probability distributions themselves that naturally give rise to generative models... You could do frequentist inference on a generative model, sure, but generative modelling seems fundamentally alien to frequentist thinking?
I am more familiar with Bayesian than frequentist stats, but given that they are mathematically equivalent, shouldn't frequentist stats have an answer to e.g. the loss function of a VAE? Or are generative machine learning inherently impossible to model for frequentist stats?
Though if you think about it, a diffusion model is somewhat (partially) frequentist.
Re: Bayesian statistics for confused data scientists
#16Earlier quoted context omitted.
I'm not well versed enough, but what would a frequentist generative model even mean? The entire generative concept implicitly assumes that parameters have probability distributions themselves that naturally give rise to generative models... You could do frequentist inference on a generative model, sure, but generative modelling seems fundamentally alien to frequentist thinking?
I am more familiar with Bayesian than frequentist stats, but given that they are mathematically equivalent, shouldn't frequentist stats have an answer to e.g. the loss function of a VAE? Or are generative machine learning inherently impossible to model for frequentist stats? Though if you think about it, a diffusion model is somewhat (partially) frequentist.
They do!
https://arxiv.org/pdf/2510.18777
But that doesn't mean a frequentist views a VAE as a generative model!
Putting it another way, Gaussian processes originated as a frequentist technique! But to a frequentist they are not generative.
Re: Bayesian statistics for confused data scientists
#17Earlier quoted context omitted.
I am more familiar with Bayesian than frequentist stats, but given that they are mathematically equivalent, shouldn't frequentist stats have an answer to e.g. the loss function of a VAE? Or are generative machine learning inherently impossible to model for frequentist stats? Though if you think about it, a diffusion model is somewhat (partially) frequentist.
They do! https://arxiv.org/pdf/2510.18777 But that doesn't mean a frequentist views a VAE as a generative model! Putting it another way, Gaussian processes originated as a frequentist technique! But to a frequentist they are not generative.
Ooh good find, thanks for the link. This will be my bedtime reading for this week :)
Re: Bayesian statistics for confused data scientists
#18I think Rafael Irizarry put it best over a decade ago -- while historically there was a feud between self-declared "frequentists" and "Bayesians", people doing statistics in the modern era aren't interested in playing sides, but use a combination of techniques originating in both camps: https://simplystatistics.org/posts/2014-10-13-as-an-applied-...
I agree... I feel like "The Elements of Statistical Learning" was possibly one of the first "postmodern" things where "well, frequentist and Bayesian are just tools in the toolbox, we now know they're not so incompatible."
After Stein's paradox it became super hard to be a pure frequentist if you didn't have your head in the sand.
Re: Bayesian statistics for confused data scientists
#19I went through grad school in a very frequentist environment. We “learned” Bayesian methods but we never used them much. In my professional life I’ve never personally worked on a problem that I felt wasn’t adequately approached with frequentist methods. I’m sure other people’s experiences are different depending on the problems you gravitate towards. In fact, I tend to get pretty frustrated with Bayesian approaches b…
I feel like I'm a polyglot here but primarily a native frequentist thinker.
I've found Bayesian methods shine in cases of an "intractible partition function".
Cases such as language models, where the cardinality of your discrete probability distribution is extremely large, to the point of intractability.
Bayesians tend to immediately go to things like Monte Carlo estimation. Is that fundamentally Bayesian and anti-frequentist? Not really... it's just that being open to Bayesian ways of thinking leads you towards that more.
Reinforcement learning also feels much more naturally Bayesian. I mean Thompson sampling, the granddaddy of RL, was developed through a frequentist lens. But it also feels very Bayesian as well.
In the modern era, we have Stein's paradox, and it all feels the same.
Hardcore Bayesians that seem to deeply hate the Kolmogorov measure theoretic approach to probability are always interesting to me as some of the last true radicals.
I feel like 99% of the world today is these are all just tools and we use them where they're useful.
Re: Bayesian statistics for confused data scientists
#20Earlier quoted context omitted.
I'm not well versed enough, but what would a frequentist generative model even mean? The entire generative concept implicitly assumes that parameters have probability distributions themselves that naturally give rise to generative models... You could do frequentist inference on a generative model, sure, but generative modelling seems fundamentally alien to frequentist thinking?
I am more familiar with Bayesian than frequentist stats, but given that they are mathematically equivalent, shouldn't frequentist stats have an answer to e.g. the loss function of a VAE? Or are generative machine learning inherently impossible to model for frequentist stats? Though if you think about it, a diffusion model is somewhat (partially) frequentist.
I guess you have me thinking more... things like Parzen window estimators or other KDEs are frequentist...
But while it's a probability distribution, to a frequentist they are estimating the fixed parameters of a distribution.
The distribution isn't generative, it just represents uncertainty - and I think that's a bit of the deep core philosophical divide between frequentists and Bayesians - you might use all the same math, but you cannot possibly think of it as being generative.