Wow this is really interesting and might be really helpful for a problem I'm working on now. I'm wondering though, what is the advantage of using a whole new programming language rather than say, a Python library?
Probabilistic Programming
11–20 of 30 posts
Re: Probabilistic Programming
#12Wow this is really interesting and might be really helpful for a problem I'm working on now. I'm wondering though, what is the advantage of using a whole new programming language rather than say, a Python library?
>I'm wondering though, what is the advantage of using a whole new programming language rather than say, a Python library?
The idea is that if you can write the program, you can sample from the generative model. Most libraries like Stan don't really provide a "Turing-complete" probabilistic language.
Daniel Roy, IIRC, studies precisely why or when you want this kind of "completeness", as well as other topics in PPL theory.
Re: Probabilistic Programming
#13Wow this is really interesting and might be really helpful for a problem I'm working on now. I'm wondering though, what is the advantage of using a whole new programming language rather than say, a Python library?
There are packages that are just Python libraries, like PyMC3: http://pymc-devs.github.io/pymc3/ It allows you to specify a Probabilistic Program with a few lines of Python code (x ~ Normal(0, 1) becomes x = pymc3.Normal('x', 0, 1)) and then run automatic inference on whatever model you dream up, it's very powerful. I always found the approach of inventing a new language to specify Probabilistic Programs like Stan does a bit odd. There, you specify the model as a string and can't interact with it or debug if something goes wrong. Disclaimer: I'm a PyMC3 developer.
Re: Probabilistic Programming
#14For those who are interested in learning more about Probabilistic Programming:
Re: Probabilistic Programming
#15I'm aware of Stan - does it cover what probabilistic programming is about or is there more to it?
There's more to it. Stan does models, as akin to statistical models. Workflow: specify model - run - evaluate results. Within that sphere they f.e. don't support all types of variables, but are fast, well-documented and under active development.
Outside of that sphere there is even more: you can make any program probabilistic / learning with something like PyMC that embed PP in a regular environment. Think bots, agents, UIs ...
Re: Probabilistic Programming
#16If you like to try a probabilistic programming language with Lisp/Scheme syntax, there's Church https://projects.csail.mit.edu/church/wiki/Church
Re: Probabilistic Programming
#17If you're interested in a probabilistic prolog, there is ProbLog [0].
Re: Probabilistic Programming
#18Heh, I just started doing some searching on probabilistic programming about an hour ago. Funny how that works. Since it's on subject here - anybody know of any good clojure libraries or resources for probabilistic programming? Anglican definitely looks good, but their intro pages use some kinda strange workflows.
I think right now for clojure, Anglican is the one furthest along. It took me some effort to get into it, with the documentation being a little bit all over the place, but I got it working for me.
Do remember that Anglican is a partial language in clojure, so you can't use all clojure functions (and I think none of the macros) within anglican's defqueries.
Re: Probabilistic Programming
#19I'm aware of Stan - does it cover what probabilistic programming is about or is there more to it?
There's more to it. Stan does models, as akin to statistical models. Workflow: specify model - run - evaluate results. Within that sphere they f.e. don't support all types of variables, but are fast, well-documented and under active development. Outside of that sphere there is even more: you can make any program probabilistic / learning with something like PyMC that embed PP in a regular environment. Think bots, agen…
PyMC is roughly equivalent to Stan. I think what you're getting at is more the applications of these statistical models. Stan can be integrated into Python via PyStan.
Re: Probabilistic Programming
#20>> It's more like Prolog: sure, it's a programming language—but it's not the
right tool for writing full-fledged software.
Uh oh. That's dangerous talk.
The problem is that once you start writing the kind of program that Prolog is best at, in Prolog, then you really, _really_ don't want to go back and write the same thing in any other language. Because most of the time it's a huge pain, and you'll probably have to implement Prolog in that other language anyway (and it's going to be a slow and buggy Prolog, but without all the good bits).
This has actually harmed the adoption of Prolog in a roundabout way, I think. Programmers are (or, well, were ... in the distant past) worried they might be stuck with it and unable to write the everyday stuff they need (REST APIs and whatnot).
If PP languages are sold in the same way, as very niche products that are only useful for a specific kind of programming, then I don't think they are ever going to be widely adopted, not even from the people who could benefit from them. Which is to say: eventually, everyone.
Instead of risking being stuck with very valuable software written in a one-trick pony sort of language, much better to take the long road and write it all in a language like C# or Java, where it may look attrocious, but at least it's the same language as the rest of your application (and you can find plenty of warm bodies to throw at it once it starts going south, to boot).