Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Best books on AI?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 95 posts

Re: Ask HN: Best books on AI?

#51

Post by curuinor »

king_magic wrote: post #37
curuinor wrote: post #13

Earlier quoted context omitted.

Less symbolic failures. There was a huge and abiding torrent of neural net stuff that dealt with evolving topologies in late 90's. I see very little of it in any way shape or form in industry or academia today, because it's a lot of computation for basically no gain. They thought that layerwise pretraining of neural nets was the way to go in 2006, before they realized that initializations, normalization, and better a…

Is pretraining really all that much of a failure? I haven't really found an authoritative answer on whether or not pretraining is worth it these days. Hinton's 2012(?) Coursera course still focuses pretty deeply on generative/layer-by-layer pretraining with RBMs but I'm just not really sure if that's fallen by the wayside today. Or maybe it's still useful only in specific circumstances?

Saxe Ganguli McClelland, 2013, about linear nets and orthogonal initialization. But then, read Li Jiao Han Weissman 2017 (maybe preprint), "Demystifying ResNet", which makes a nice claim about the niceness being conditioning of Hessian at init.

Tldr: it's good conditioner but you can do better ab initio

Re: Ask HN: Best books on AI?

#52

Post by curuinor »

nextos wrote: post #35
arihant wrote: post #26

You cannot really go practical in AI without academic rigor. You can do recipes of what's already been done using a TensorFlow book, but that's how far one can go. If one is serious in getting in AI today, a great way to do is read the following books, in order: 1. AI: A Modern Approach by Stuart Russell and Peter Norvig. 2. Deep Learning by Ian Goodfellow and Yoshua Bengio. It is amazing how approachable both books…

I love Paradigms of Artificial Intelligence by Peter Norvig much more than AIMA, which I found excessively encyclopedic and shallow. While some will argue it is dated, I think it presents many timeless ideas that will get in vogue soon with little tweaks to their inference schemes. Same for The Art of Prolog.

If you're shipping anything after 2010, you're not going to get within an order of magnitude within state of the art with that book (PAI), unfortunately.

There's basically no numerics in that book about anything that'll past muster at NIPS or ICML nowadays or would be shipped by one of the big corporate AI labs, I'm sorry to say.

Re: Ask HN: Best books on AI?

#53

Post by curuinor »

gugagore wrote: post #41
flor1s wrote: post #17

Earlier quoted context omitted.

I think to be successful at machine learning you also need a good understanding of calculus, besides probability and linear algebra.

but perhaps not too much: https://news.ycombinator.com/item?id=9189553

Strang's complaint is that there's too little linear algebra. This is true. This doesn't overshadow the fact that you're not going to get out of using some partial derivatives in neural net land (and many other AI subfields).

Re: Ask HN: Best books on AI?

#54

Post by andyczerwonka »

arihant wrote: post #26

You cannot really go practical in AI without academic rigor. You can do recipes of what's already been done using a TensorFlow book, but that's how far one can go. If one is serious in getting in AI today, a great way to do is read the following books, in order: 1. AI: A Modern Approach by Stuart Russell and Peter Norvig. 2. Deep Learning by Ian Goodfellow and Yoshua Bengio. It is amazing how approachable both books…

I have both, and I agree these are great. I really appreciate the style of the Goodfellow book, it's very approachable.

Re: Ask HN: Best books on AI?

#55

Post by dominotw »

krosaen wrote: post #30

Some on this thread have recommended Norvig's PAIP, but that's kind of an old school AI book in that it focuses on heuristic search and logic (implementing prolog in lisp at one point, very impressive stuff actually); but is lacking any coverage of statistical machine learning, which is the approach that underlies most of the cool stuff these days. It's still a great book, but I'd instead recommend a path that focuse…

Hi Karl, I just read through your very intersting post about the sabbatical. I 've been thinking about this a while too. I see that you are currently in a A.I role at umich, I was wondering you have written anything about how you went about actually landing a role in this particular field.

Thank you.

Re: Ask HN: Best books on AI?

#56

Post by flor1s »

gugagore wrote: post #41
flor1s wrote: post #17

Earlier quoted context omitted.

I think to be successful at machine learning you also need a good understanding of calculus, besides probability and linear algebra.

but perhaps not too much: https://news.ycombinator.com/item?id=9189553

What is too much though? Backpropagation uses derivatives, some filters in Computer Vision use multivariate calculus. If you want to have a thorough understanding then calculus is necessary. That said, Andrew Ng was quite good at avoiding calculus in his Machine Learning MOOC, and for applied machine learning I guess calculus is not that important.

A great place to study about math is www.khanacademy.org, they have courses on calculus, probability/statistics and linear algebra.

Re: Ask HN: Best books on AI?

#57

Post by nojvek »

taktoa wrote: post #47
JamilD wrote: post #39

Earlier quoted context omitted.

You don't need to focus on just one side or the other. AlphaGo wasn't made by just hacking together a couple neural nets, it built on heuristic search, MCTS, and the "old-school" AI. I agree that basic statistics + Bishop's book is a great way to start getting into machine learning -- but AI is a much broader field than that.

"old-school" AI is just a pretentious name for algorithms/graph theory/combinatorics, plus a bit of (now very outdated) PLT mixed in. "new-school" AI (machine learning) is just a more pretentious name for statistics/control theory/randomized algorithms.

What is to say the human brain doesn't engage in graph combinotrics with some learned heuristics.

Some of the old ideas with enough computation power are actually pretty amazing.

Re: Ask HN: Best books on AI?

#58

Post by pvinis »

nextos wrote: post #35
arihant wrote: post #26

You cannot really go practical in AI without academic rigor. You can do recipes of what's already been done using a TensorFlow book, but that's how far one can go. If one is serious in getting in AI today, a great way to do is read the following books, in order: 1. AI: A Modern Approach by Stuart Russell and Peter Norvig. 2. Deep Learning by Ian Goodfellow and Yoshua Bengio. It is amazing how approachable both books…

I love Paradigms of Artificial Intelligence by Peter Norvig much more than AIMA, which I found excessively encyclopedic and shallow. While some will argue it is dated, I think it presents many timeless ideas that will get in vogue soon with little tweaks to their inference schemes. Same for The Art of Prolog.

Art of Prolog was over of the best books I've read. As Prolog itself is one of my favorite languages. I wish there were more of both.

Re: Ask HN: Best books on AI?

#59

Post by stesch »

nextos wrote: post #35
arihant wrote: post #26

You cannot really go practical in AI without academic rigor. You can do recipes of what's already been done using a TensorFlow book, but that's how far one can go. If one is serious in getting in AI today, a great way to do is read the following books, in order: 1. AI: A Modern Approach by Stuart Russell and Peter Norvig. 2. Deep Learning by Ian Goodfellow and Yoshua Bengio. It is amazing how approachable both books…

I love Paradigms of Artificial Intelligence by Peter Norvig much more than AIMA, which I found excessively encyclopedic and shallow. While some will argue it is dated, I think it presents many timeless ideas that will get in vogue soon with little tweaks to their inference schemes. Same for The Art of Prolog.

I had the feeling it was more of a Common Lisp book than AI book.

Re: Ask HN: Best books on AI?

#60

Post by [deleted] »

curuinor wrote: post #51
king_magic wrote: post #37

Earlier quoted context omitted.

Is pretraining really all that much of a failure? I haven't really found an authoritative answer on whether or not pretraining is worth it these days. Hinton's 2012(?) Coursera course still focuses pretty deeply on generative/layer-by-layer pretraining with RBMs but I'm just not really sure if that's fallen by the wayside today. Or maybe it's still useful only in specific circumstances?

Saxe Ganguli McClelland, 2013, about linear nets and orthogonal initialization. But then, read Li Jiao Han Weissman 2017 (maybe preprint), "Demystifying ResNet", which makes a nice claim about the niceness being conditioning of Hessian at init. Tldr: it's good conditioner but you can do better ab initio

[deleted]
Post reply on HN