Deep Learning vs. Probabilistic Graphical Models vs. Logic
quantombone.blogspot.com
Deep Learning vs. Probabilistic Graphical Models vs. Logic
1–10 of 28 posts
Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic
#2After learning PGMs, I find that I've almost completely eschewed first order logic for my own personal everyday reasoning. Arguments based on logical formulations require that the propositions are not leaky abstractions, and for most problem domains (i.e. not physics), there are going to be so many exceptions that I find very few cases where I can rely on first order logic. The softness of PGMs, and ideas like "explaining away" [1] come in quite handy. And after learning some of Pearl's (and others) formulation of causality as graphical models, I understand much better why counterfactual reasoning is so error-prone.
Further, PGMs have the advantage over deep networks in that they are highly explainable, and you can go back and look at the chain of reasoning. For some problem domains, this part is more important than prediction accuracy.
[1] http://www.cs.ubc.ca/~murphyk/Bayes/bnintro.html#explainaway
Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic
#3After learning PGMs, I find that I've almost completely eschewed first order logic for my own personal everyday reasoning. Arguments based on logical formulations require that the propositions are not leaky abstractions, and for most problem domains (i.e. not physics), there are going to be so many exceptions that I find very few cases where I can rely on first order logic. The softness of PGMs, and ideas like "expla…
The power of logic is that a few well chosen domain specific clauses can reduce the problem dimensionality dramatically.
If you are building a robot, even if the mechanics are not really newtonian, modeling the system mechanically can get a model much closer to the underlying manifold, reduce training set size and improve generalizability. So I don't think the old way of doing things should be thrown out. They got pretty near the right answers, and we should use newer methods to just fill in the gap between theory and practice.
E.g. pre-train a DBN using an analytical model and later adjust it on real data.
Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic
#4I do not think logic-based learning is dead. It just smells a bit funny.
In the vein of the papers "From machine learning to machine reasoning" and "Text understanding from scratch" I expect a "First-order logic understanding from scratch" to follow naturally.
Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic
#5After learning PGMs, I find that I've almost completely eschewed first order logic for my own personal everyday reasoning. Arguments based on logical formulations require that the propositions are not leaky abstractions, and for most problem domains (i.e. not physics), there are going to be so many exceptions that I find very few cases where I can rely on first order logic. The softness of PGMs, and ideas like "expla…
Very curious to know
1. Where you learnt PGMs
2. How you made it part of your 'personal everyday' toolkit
Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic
#6I have worked with Prolog a bit so FOL is somewhat familiar (I wouldn't call myself an expert by any means). FOL is quite an amazing tool to reduce the problem space in well defined environments. I enjoy board games and rules based FOL AIs are pretty well suited in that domain. Modelling non-trivial domains as a set of rules is pretty tough though (+Gödel applies). Creating game like structures for everyday stuff is one of my remaining AI research interests (the idea being that expert knowledge can somehow be modelled as AIs that compete in the game and thus be made comparable).
The "Inductive Logic Programming" chapter in "Prolog Programming for AI" (best intro Prolog book imo) is very interesting and has lead to a couple of entries in my todo list :)
Non-Standard logics are also very fascinating.
I love "AI A Modern Approach" but the chapter on PGMs wasn't the best in my opinion. I think the dentist example just bothered me/it wasn't all that obvious how useful they really are. Thankfully the book is amazing and they provide plenty of references to move on :) That being said I think PGMs are immensely powerful and my gut says this approach is the one that I like the best.
Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic
#7Logic based AI is definitely taking a backseat to data driven methods in the current environment but dead is a gross exaggeration. There are a large class of problems for which heuristic search in logic domains is the most performant technique, and a significant class of problems where SAT solvers are feasible solutions. Many of them are real world examples, rather than academic problems. Also, many techniques are evolved or emerging for systems of logic that can handle uncertainty. I've done work on hybrid systems combining rule-based systems with data-based systems (the typical process takes a rule-based system as a starting point and evolves it towards a pure data system as the data sets get large enough). However, starting with a rule-based system is actually a good approach for most start-ups when you don't have enough data to get performant models.
Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic
#8After learning PGMs, I find that I've almost completely eschewed first order logic for my own personal everyday reasoning. Arguments based on logical formulations require that the propositions are not leaky abstractions, and for most problem domains (i.e. not physics), there are going to be so many exceptions that I find very few cases where I can rely on first order logic. The softness of PGMs, and ideas like "expla…
Very curious to know 1. Where you learnt PGMs 2. How you made it part of your 'personal everyday' toolkit Always interested in improving my thought processes...
I don't know how he learned, but I studied it through the very demanding, and worth every second, course from Coursera:
Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic
#9Earlier quoted context omitted.
Very curious to know 1. Where you learnt PGMs 2. How you made it part of your 'personal everyday' toolkit Always interested in improving my thought processes...
I don't know how he learned, but I studied it through the very demanding, and worth every second, course from Coursera: https://www.coursera.org/course/pgm
I read somewhere that this is one of the hardest Coursera courses.
Re: Deep Learning vs. Probabilistic Graphical Models vs. Logic
#10Earlier quoted context omitted.
I don't know how he learned, but I studied it through the very demanding, and worth every second, course from Coursera: https://www.coursera.org/course/pgm
I read somewhere that this is one of the hardest Coursera courses.
Indeed, it is demanding, but fascinating and very well taught.
Too bad they haven't offered it since 2013. I didn't finished it by them for personal reasons :c/