Why Probabilistic Programming Matters
zinkov.com
Why Probabilistic Programming Matters
1–10 of 15 posts
Re: Why Probabilistic Programming Matters
#2The question I'd have about creating a new programming language for a new technique is what does the syntax give you to compensate for the costing of learning a new language?
Re: Why Probabilistic Programming Matters
#3The point with probabilistic programming is you are able to explore slightly more complex models very easily.
Coincidentally, a colleague at lunch yesterday made exactly this observation about Stan, a probabilistic programming language he is using (http://mc-stan.org/).
Stan's notation is very similar to, if not the same as, that shown here. In Stan, those model{} statements are combined with data{} and parameter{} statements that plug in values of interest. The algorithmic details appear to get specified by the command-line invocation.
I found the Stan user guide a bit more enlightening than this essay. The essay shows a bunch of models, but it has very little discussion about what the models mean, or how they express the ideas alluded to in the short introductory paragraphs. The Stan UG does a marginally better job of showing why the model looks the way it does, and relating it to the math they're trying to solve. Still, like this essay, the exposition is definitely pitched toward somebody who already knows the field (unfortunately, not me).
Re: Why Probabilistic Programming Matters
#4The question I'd have about creating a new programming language for a new technique is what does the syntax give you to compensate for the costing of learning a new language?
This is not an issue of syntax. You can have probabilistic languages with existing syntax--Church, for example, uses Scheme syntax[1].
[1]: http://projects.csail.mit.edu/church/wiki/Mixture_models
You can even embed your new language in another, well-known general-purpose language, letting you reuse the host language's syntax. Haskell is great for this sort of work.
This is entirely a question of semantics. And what do probabilistic semantics give you? They make writing complex models for data much more natural. These languages allow you to abstract away much of the incidental complexity of specifying these models, making the logic clearer. In turn, this simplifies reasoning about the problem and working on a solution.
More generally, this is the motivation behind most domain-specific languages. The core idea is to express yourself in a way that fits the domain you're working in. For machine-learning, you want to express your program in terms of probability distributions. For writing parsers, you want to express yourself in terms of CFGs. As people like to repeat, a program written more for people to read than computers to execute, and reading something in terms relevant to the domain is easier than trying to force the domain to fit an existing programming language and paradigm.
Re: Why Probabilistic Programming Matters
#5The point with probabilistic programming is you are able to explore slightly more complex models very easily. Coincidentally, a colleague at lunch yesterday made exactly this observation about Stan, a probabilistic programming language he is using ( http://mc-stan.org/ ). Stan's notation is very similar to, if not the same as, that shown here. In Stan, those model{} statements are combined with data{} and parameter{}…
I had never heard of probabilistic programming languages and was intrigued but unfortunately also rather confused by the submission. So thank you for the excellent pointer.
Re: Why Probabilistic Programming Matters
#6i am confused about whether it's best to post here on in a separate thread, so apologies for doing both, but this provoked me to look for a good intro to probabilistic programming and i found http://research.microsoft.com/en-us/um/cambridge/projects/in... which is a very easy read.
Re: Why Probabilistic Programming Matters
#7Re: Why Probabilistic Programming Matters
#8The point with probabilistic programming is you are able to explore slightly more complex models very easily. Coincidentally, a colleague at lunch yesterday made exactly this observation about Stan, a probabilistic programming language he is using ( http://mc-stan.org/ ). Stan's notation is very similar to, if not the same as, that shown here. In Stan, those model{} statements are combined with data{} and parameter{}…
I had never heard of probabilistic programming languages and was intrigued but unfortunately also rather confused by the submission. So thank you for the excellent pointer.
Stan aims to be the successor to BUGS, and it is made in part by the great Bayesian Andrew Gelman. Nevertheless, BUGS was the de facto language for probabilistic modelling for many years, and you may find better documentation for it.
Re: Why Probabilistic Programming Matters
#9i am confused about whether it's best to post here on in a separate thread, so apologies for doing both, but this provoked me to look for a good intro to probabilistic programming and i found http://research.microsoft.com/en-us/um/cambridge/projects/in... which is a very easy read.
I only got partway into http://projects.csail.mit.edu/church/wiki/Probabilistic_Mode... but mean to come back -- it looks worthwhile.
Re: Why Probabilistic Programming Matters
#10The point with probabilistic programming is you are able to explore slightly more complex models very easily. Coincidentally, a colleague at lunch yesterday made exactly this observation about Stan, a probabilistic programming language he is using ( http://mc-stan.org/ ). Stan's notation is very similar to, if not the same as, that shown here. In Stan, those model{} statements are combined with data{} and parameter{}…
I had never heard of probabilistic programming languages and was intrigued but unfortunately also rather confused by the submission. So thank you for the excellent pointer.
Anyway, I wrote this post as a precursor to a more substantial set of writings explaining how to practically use these upcoming languages. Most of what I am trying to show here is that while we use specialized algorithms for most of our machine learning at this moment, it is only a temporary solution before it becomes feasible to just specify your models in one of these languages. The point of the code is to show that many really complicated models with high specialized libraries for just them is less than 10 lines of code in this framework.
For a really good introduction to this topic I recommend of Josh Tenenbaum's Growing a Mind talks, http://videolectures.net/aaai2012_tenenbaum_grow_mind/
I never realized this article would be read by regular folks and am incredibly confused why something I wrote 6 months ago is suddenly all over the internet now.