An Introduction to Probabilistic Graphical Models (2003) [pdf]
people.eecs.berkeley.edu
An Introduction to Probabilistic Graphical Models (2003) [pdf]
1–10 of 37 posts
Re: An Introduction to Probabilistic Graphical Models (2003) [pdf]
#2PGM seems to me harder than neural nets, but the trend in the last couple of years is to include probabilities in neural nets, so they're hot.
Re: An Introduction to Probabilistic Graphical Models (2003) [pdf]
#3PGM seems to me harder than neural nets, but the trend in the last couple of years is to include probabilities in neural nets, so they're hot.
Not an expert but PGMs are mostly a superset of NN, so it's kinda understandable.
Re: An Introduction to Probabilistic Graphical Models (2003) [pdf]
#4PGM seems to me harder than neural nets, but the trend in the last couple of years is to include probabilities in neural nets, so they're hot.
I used "Probabilistic Graphical Models" By Koller/Friedman
[0] https://www.amazon.com/Probabilistic-Graphical-Models-Princi...
Re: An Introduction to Probabilistic Graphical Models (2003) [pdf]
#5PGM's are great, but my experience from Koller's course is that it is very hard to identify cases where they can be used.
Part of the reason is that you need a-priori knowledge of the causal relationships (coarse grained I.e direction) between your variables.
Presumably if you're doing ML you don't know those causal relationships to begin with.
Particularly good fits are things like physics where laws are known.
Re: An Introduction to Probabilistic Graphical Models (2003) [pdf]
#6There's an excellent course on PGM by Koller on Coursera. My friend took it and now he's a PGM evangelist. If you are wondering where PGM lies in the spectrum of machine learning, you should research the difference between generative and discriminate modeling. We have been driven to PGM to solve our ML problem that was hard to frame as A NN. Mainly because we had some priors we needed to encode to make the problem tractable. It reminds me a little of heuristics in search.
The person I'm talking to: an early ML student.
Re: An Introduction to Probabilistic Graphical Models (2003) [pdf]
#7This is the best textbook on graphical models, also from Jordan but later (2008): https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_F.... It also covers some general theory of variational inference. Source: I worked on PGMs in grad school.
Re: An Introduction to Probabilistic Graphical Models (2003) [pdf]
#8A few comments have mentioned neural nets in this post. adamnemecek mentions in this thread that PGMs are a superset of neural networks, and
and Thomas Wiecki has a few excellent blog posts on creating bayesian neural networks using pymc3.[0][1][2] If you're curious about how these two concepts can be brought together I highly recommend reading through these three posts.
[0] http://twiecki.github.io/blog/2016/06/01/bayesian-deep-learn...
[1] http://twiecki.github.io/blog/2016/07/05/bayesian-deep-learn...
[2] http://twiecki.github.io/blog/2017/03/14/random-walk-deep-ne...
Re: An Introduction to Probabilistic Graphical Models (2003) [pdf]
#9PGM's are great, but my experience from Koller's course is that it is very hard to identify cases where they can be used. Part of the reason is that you need a-priori knowledge of the causal relationships (coarse grained I.e direction) between your variables. Presumably if you're doing ML you don't know those causal relationships to begin with. Particularly good fits are things like physics where laws are known.
A heuristic guide...If you have more data than knowledge about the domain, and you want to do classification/prediction, then NNs are a good.
PGMs are good if you have knowledge that is important to encode; you need modularity i.e. you don't want to embed priors in the model; you have strong causal relationships e.g. diagnostic models with 'explaining away'; you want to integrate value of information e.g. what test should I do next to resolve uncertainty.
You can mix ML with knowledge in PGMs. The downside is the computational complexity of inference which is NP-hard for both exact and approximate inference, although you can identify the complexity of the model at design time and make the appropriate modelling trade-offs.
Re: An Introduction to Probabilistic Graphical Models (2003) [pdf]
#10PGM seems to me harder than neural nets, but the trend in the last couple of years is to include probabilities in neural nets, so they're hot.
Not an expert but PGMs are mostly a superset of NN, so it's kinda understandable.
What is the connection other than "they both have graphs somewhere in them"?
I sort of see what you mean since NNs transport information across a graph in a straightforward (non-loopy) way, and PGMs can propagate information in crazy (neverending loops) ways when doing belief propagation...