A visual introduction to Gaussian Belief Propagation • Hacker News Board

Re: A visual introduction to Gaussian Belief Propagation

#11

Post by 123pie123 » Tue, Aug 31, 2021, 5:54 PM UTC

youssefabdelm wrote: post #9

vignesh_warar wrote: post #4

What a great article - and the way it's presented is insanely great. It reminds me of a talk given by Bret Victor https://youtu.be/oUaOucZRlmE?t=1175 (Media for Thinking the Unthinkable)

Was just about to say it reminded me of Bret Victor. But someone needs to synthesize these technical articles with the field of General Semantics ( https://www.youtube.com/playlist?list=PLaoJIXlyLvLkMQUtbiTi1... ) to make it even clearer for the curious complete beginner to mathematical/technical thinking/approaches. A deeper awareness of all the jargon one is using + deep intuitive layman explanation would be valuab…

that's neat, do you know what the drawing / visualization program is called?

Re: A visual introduction to Gaussian Belief Propagation

#12

Post by epistasis » Tue, Aug 31, 2021, 7:00 PM UTC

wiz21c wrote: post #6

I'm a bit of noob in the matter but : FTA: "Looking to the future, we anticipate a long-term trend towards a "hardware jungle" of parallel, heterogeneous, distributed and asynchronous computing systems which communicate in a peer-to-peer manner. " Does the HN crowd think it's right ?

As somebody who has used belief propagation a lot, before the current neural network renaissance, I'm not yet optimistic of that. (However, I have been wrong enough in the past to be somewhat humble about my ability to predict these things!)

As I see it, the current big machine learning models use data that can't fit well on a single machine, or training regimes that can't fit onto a single machine, but the end model still fits on a single GPU or box.

What I understand them to be advocating for here, is a model that is bigger than can fit on a single chip, that can use belief propagation to scale to that number of variables/size. I don't yet know what sort of applications that could address, so that's my primary point of skepticism about this sort of huge model being developed. BUT, my lack of imagination is not much of an argument on its own, it's merely the absence of an argument, not an actual argument against such large models being useful.

Re: A visual introduction to Gaussian Belief Propagation

#13

Post by shoo » Tue, Aug 31, 2021, 9:52 PM UTC

wiz21c wrote: post #7

Question to HN: is there somewhere a dictionary explaining and relating terms, in applicable way, that everyone uses without ever explaining them, such as "priors", "posteriors", "marginals", "belief", "marginal distribution", etc.

if you're willing to sink time into a lecture series, you may enjoy Richard McElreath's introductory lectures on bayesian inference: https://www.youtube.com/watch?v=_NEMHM1wDfI

Re: A visual introduction to Gaussian Belief Propagation

#14

Post by nextos » Wed, Sep 01, 2021, 12:11 AM UTC

wiz21c wrote: post #7

Question to HN: is there somewhere a dictionary explaining and relating terms, in applicable way, that everyone uses without ever explaining them, such as "priors", "posteriors", "marginals", "belief", "marginal distribution", etc.

Regression and Other Stories by Andrew Gelman et al. is IMHO the best introduction to Bayesian reasoning without being overly technical, but still maintaining rigor and providing lots of insights about how statistics is done in practice.