A Sober Look at Bayesian Neural Networks

Also, I'll admit that messing up Bayes rule in a blog post criticizing Bayesian Neural Netowrks is pretty comical. Should have taken the time to proof read the whole thing.

The thing he's trying to explain is how those things interact and what their behavior is.

>> But there is one core problem with the Bayesian framework. In practice, we never have access to the prior distribution Pr(f)! Who could ever claim to know the real-world distribution of functions that solve classification tasks? Not us, and certainly not Bayesians. Instead, BNNs simply choose an arbitrary prior distribution q(f) over functions, and Bayesian inference is performed to compute q(f∣D). The question of whether q(f) is close to the true distribution Pr(f) is swept under the rug.

This is true but it's also nothing new: it's the standard PAC-Learning assumption that the examples (the dataset) are drawn from the same distribution as the target theory (the real-world distribution).

This assumption and the complete impossibility to verify it in practice is not unique to Bayesian Neural Nets. It is true for _every_ machine learning algorithm.

And this is certainly no surprise for machine learning researchers (or, if it is, it is really concerning that it is). So the done thing in machine learning research is to demonstrate that, under PAC-Learning assumptions, a certain technique or algorithm _can_ correctly identify a hypothesis that approximates a "true" function to within some amount of error.

I mean to say, when people publish papers reporting a new SOTA on such-and-such dataset, they are not really claiming that their technique somehow finds the "true" distribution of the real-world process that generated the data in their dataset. They're claiming "we can correctly classify instances in this dataset and if PAC-Learning assumptions hold, this technique should also work in real-world data from the same domain".

Of course this is often left implicit- and the article makes me wonder to what extent this is because researches tend to forget or even :gasp: ignore it completely. A disturbing thought.

> So viewed through this lens, BNNs with arbitrary priors are nothing more than an architectural decision. A BNN is just a neural network that maps its input to a distribution over outputs; the prior is just a hyperparameter of the model. Just making the network Bayesian bought us nothing. It will only be helpful if we find a good prior, and validate that we are actually doing accurate inference. If you personally believe that exploring this space of priors (similar to exploring the space of architectures or hyperparameters) is particularly promising, then that is a good reason to keep working on BNNs.

Isn't this overstating it a bit... Making the network Bayesian (even if the priors are generalization-agnostic) still bought us something: after training we have a posterior distribution over weights that contains both weights that generalize and weights that don't generalize. That's better than being unlucky and only having weights that don't generalize at the end of training. I'm not saying this advantage is easy to achieve in practice, or benefit from. But it's not nothing.

Also Bayesian inference as a framework has a distinct advantage over the usual aimless architectural twiddling that so much "research" seems to focus on: it's intellectually/mathematically sound.