Deep Learning vs. Probabilistic Graphical Models vs. Logic • Hacker News Board

Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic

#11

Post by denniskane » Sat, Apr 11, 2015, 2:13 PM UTC

TimPC wrote: post #7

Logic based AI is definitely taking a backseat to data driven methods in the current environment but dead is a gross exaggeration. There are a large class of problems for which heuristic search in logic domains is the most performant technique, and a significant class of problems where SAT solvers are feasible solutions. Many of them are real world examples, rather than academic problems. Also, many techniques are ev…

I am starting to get a (what I guess you could call mostly logic-based) AI system off the ground. The first thing I needed to do was create a serious environment that could host it. The necessary attributes of this environment are these:

1) It is fully web-based, and thus it is technically "on" the web, and accessible by everyone

2) It does not deal with any "modern web appy"-type meta-frameworks, and thus it is not, so-to-speak, "of" [what most of today's web developers would call] the web, and it therefore has no dependency issues to hold back its development

3) It is essentially a working Unix-like development environment, complete with a standard(ish) shell.

If you use Chrome, you can find it at https://www.urdesk.net

To go directly to the AI, just follow this link: https://www.urdesk.net/desk?intro=bertie

As far as the issue of data vs. rules is concerned, I don't know what it would even mean for a system to be purely one or the other.

Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic

#12

Post by sonabinu » Sat, Apr 11, 2015, 2:48 PM UTC

aylons wrote: post #10

rndn wrote: post #9

Earlier quoted context omitted.

I read somewhere that this is one of the hardest Coursera courses.

Indeed, it is demanding, but fascinating and very well taught. Too bad they haven't offered it since 2013. I didn't finished it by them for personal reasons :c/

15 - 20 hours a week is pretty demanding!

Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic

#13

Post by nmrm » Sat, Apr 11, 2015, 4:12 PM UTC

Of course if you conflate logic with just traditional first order logic then all the interesting fruit has been picked.

But there are many logics that can be used to reason about stochastic and probabilistic dynamics.

Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic

#14

Post by murbard2 » Sat, Apr 11, 2015, 4:18 PM UTC

tlarkworthy wrote: post #3

epistasis wrote: post #2

After learning PGMs, I find that I've almost completely eschewed first order logic for my own personal everyday reasoning. Arguments based on logical formulations require that the propositions are not leaky abstractions, and for most problem domains (i.e. not physics), there are going to be so many exceptions that I find very few cases where I can rely on first order logic. The softness of PGMs, and ideas like "expla…

The power of logic is that a few well chosen domain specific clauses can reduce the problem dimensionality dramatically. If you are building a robot, even if the mechanics are not really newtonian, modeling the system mechanically can get a model much closer to the underlying manifold, reduce training set size and improve generalizability. So I don't think the old way of doing things should be thrown out. They got pr…

Satisfying a complicated constraint is not much easier than sampling from a "thin" manifold.

Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic

#15

Post by mtraven » Sat, Apr 11, 2015, 4:22 PM UTC

aylons wrote: post #10

rndn wrote: post #9

Earlier quoted context omitted.

I read somewhere that this is one of the hardest Coursera courses.

Indeed, it is demanding, but fascinating and very well taught. Too bad they haven't offered it since 2013. I didn't finished it by them for personal reasons :c/

Try the BOOC: http://mitpress.mit.edu/books/probabilistic-graphical-models

Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic

#16

Post by eli_gottlieb » Sat, Apr 11, 2015, 5:21 PM UTC

You should really look into the emerging field of probabilistic programming. Avi Pfeffer has a nice book out on it, Practical Probabilistic Programming (or at least, you can get PDFs by pre-ordering). It basically expands the PGM way of reasoning to Turing-complete domains, and "hides" the problem of coding custom inference algorithms by making them parts of the language runtime.

My personal prediction is that once we get good at learning whole probabilistic programs from data rather than just inferring free numerical parameters from data, this is going to become the dominant mode of machine reasoning.

Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic

#17

Post by waterlesscloud » Sat, Apr 11, 2015, 5:28 PM UTC

rndn wrote: post #9

aylons wrote: post #8

Earlier quoted context omitted.

I don't know how he learned, but I studied it through the very demanding, and worth every second, course from Coursera: https://www.coursera.org/course/pgm

I read somewhere that this is one of the hardest Coursera courses.

It's definitely the hardest one I've taken there. Most of the difficulty comes from the density of the lectures. She moves fast and takes it for granted that you're piecing everything together as you go. You're probably not, but at least you can go back and watch it again if necessary!

Hinton's Neural Network class was very challenging for me too, mostly because many of the concepts were unfamiliar to me. But again, I could re-watch whatever I needed to in order to get it.

Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic

#18

Post by mizzao » Sat, Apr 11, 2015, 5:40 PM UTC

A good summary of why probabilistic models are important is this article by the legendary Chris Bishop: http://research.microsoft.com/en-us/um/people/cmbishop/downl...

Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic

#19

Post by cafebeen » Sat, Apr 11, 2015, 5:53 PM UTC

An interesting thing about probability is that it's essentially a "softened" version of logical reasoning, so maybe it's more fair to say that logic-based AI was generalized:

http://bayes.wustl.edu

Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic

#20

Post by logn » Sat, Apr 11, 2015, 6:01 PM UTC

Don't under-estimate the power of single layer neural networks--classifiers. They're much cheaper to train effectively and avoid over-fitting. Also, I've had good results using multiple classifiers that essentially cast votes and adding on hand-crafted heuristics to look through the top vote getters.