Probabilistic Programming & Bayesian Methods for Hackers

This is fantastic. Both in terms of content, but also in terms of delivery.

I've been toying with ideas relating to some kind of text-book killing general publishing platform for a while, but it's not something I'll implement (in the next 5 years anyway). Obviously people are doing this kind of thing already, but this is certainly the closest implementation to the ideas I've been thinking about that I've seen.

The "understanding-first" approach is so right, especially with a topic like Bayesian methods where the logic makes _so much sense_, and yet is so easy to miss by brute-force memorization of equations. IPython notebook is a great choice. I've only skimmed the first couple of chapters but cloned and will definitely spend some time with this.

This seems inspired by the awesome Yudkowsky article that comes from a similar (introductory, not programming-specific) place: http://yudkowsky.net/rational/bayes/

For anyone that hasn't used IPython notebook and is interested in scientific computing in python, you need to check it out. The ability to mix prose and live python, with effortless plotting, storable in git, sharable with links or nbviewer, is just magic. That probably seems mundane, but for things like exploratory data analysis across a team, it's a game-changer. Another staple in this stack is Pandas ( http://pandas.pydata.org/ )

I read a little of this book when it was on Hacker News a few months ago, but only half a chapter. Did anyone read through the whole book and do you have a review?

As a researcher with MCMC interests, I agree with the grandparent post. Used as technical jargon "probabilistic programming" tends to mean: specify a model using a programming language, a compiler then works out how to do inference in that model, and writes the inference code for you.

PyMC is a toolkit that makes it easier to write inference code for a wide range of models, but isn't as automatic as the field of probabilistic programming promises.

As it says on the linked site, it lists more than probabilistic programming systems, and PyMC falls into the latter categories of things it lists:

Below we have compiled a list of probabilistic programming systems including languages, implementations/compilers, as well as software libraries for constructing probabilistic models and toolkits for building probabilistic inference algorithms.

I also like "Think Bayes" even though it is "just" a book. http://www.greenteapress.com/thinkbayes/