Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What are the foundational texts for learning about AI/ML/NN?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 114 posts

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#31

Post by mtlmtlmtlmtl »

KRAKRISMOTT wrote: post #6

Haugeland is GOFAI/cognitive science, not directly relevant to modern machine learning variety of models unless you are doing reinforcement learning or trees stuff (hey poker/chess/Go bots are pretty cool!). Russel and Norvig are the typical introductory textbooks for those. Marks and Haykins are all severely out of date (they have solid content, but they don't have the same scale of modern deep learning which has ma…

A quick point about the "tree stuff" and Norvig&Russell:

While it does cover minimax trees, alphabeta etc, it only really provides a very brief overview. The book is more of an overview of the AI/ML fields as a whole. Game playing AI is dense with various game-specific heuristics that the book scarcely mentions.

Not sure about books, but the best resource I've found on at least chess AI is chessprogramming.org, then just ingesting the papers from the field.

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#32

Post by p1esk »

mfrieswyk wrote: post #7
raz32dust wrote: post #3

I personally consider Linear algebra to be foundational in AI/ML. Intro to Linear algebra, Gilbert Strang. And his free course on MIT OCW is fantastic too. While having strong mathematical foundation is useful, I think developing intuition is even more important. For this, I recommend Andrew Ng's coursera courses first before you dive too deep.

I never took beyond Precalculus in school, thanks for the tip!

Oh, most recommendations here assume stem college math knowledge. You should become comfortable with calculus, linear algebra, and probability/stats - those are the foundations of ML.

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#33

Post by PartiallyTyped »

bradreaves2 wrote: post #15

This is off the beaten path, but consider Abu-Mostafa et al.'s "Learning from Data". https://www.amazon.com/Learning-Data-Yaser-S-Abu-Mostafa/dp/... I adore PRML, but the scope and depth is overwhelming. LfD encapsulates a number of really core principles in a simple text. The companion course is outstanding and available on EdX. The tradeoff is that LfD doesn't cover a lot of breath in terms of looking at specific a…

LfD is a great book to get people to think about complexity classes and model families. We used that in my grad course and I can recommend it.

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#34

Post by viscanti »

raz32dust wrote: post #3

I personally consider Linear algebra to be foundational in AI/ML. Intro to Linear algebra, Gilbert Strang. And his free course on MIT OCW is fantastic too. While having strong mathematical foundation is useful, I think developing intuition is even more important. For this, I recommend Andrew Ng's coursera courses first before you dive too deep.

Strang is great but he covers a lot of things that don't have much carryover to AI/ML and doesn't really cover things like Jacobians which do. Maybe there's something more useful for someone who is only learning Calculus and Linear Algebra for AI/ML than what Strang teaches.

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#35

Post by kevinskii »

softwaredoug wrote: post #2

"Introduction to Statistical Learning" - https://www.statlearning.com/ (there's also "Elements of Statistical Learning" which is a more advanced version) AI: A Modern Approach - https://aima.cs.berkeley.edu/

I agree. I read the first edition to Intro to Statistical Learning and it went into just the right level of mathematical depth. The authors also have Youtube lectures that accompany the chapters, and these are a great reinforcement of the material.

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#36

Post by sillysaurusx »

bjornsing wrote: post #27

It’s probably a bit off the beaten path, but I can highly recommend Probability Theory, The Logic of Science, by E. T. Jaynes. In the opening chapter Jaynes describes a hypothetical system he calls “The Robot”. He then lays out the mathematics of the “The Robot’s” thinking in detail: essentially Bayesian probability theory. This is the best summary of an ideal ML/AI system I’ve come across. It’s also very philosophic…

I'm so sad the editor chose not to publish Jaynes' C snippets because "they were too cryptic." They would've helped clarify the ideas greatly.

It's a good book, but I don't know how it's related to ML. My own answer would be "Just do it." Find an ML project you like and start tinkering around. But everyone learns differently, so maybe there's a book that can replace experience.

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#37

Post by TaupeRanger »

There are none anymore. We now know that throwing a bunch of bits into the linear algebra meat grinder gets you endless high quality art and decent linguistic functionality. The architecture of these systems takes maybe a week to deeply understand, or maybe a month for a beginner. That's really it. Everything else is obsolete or no longer applicable unless you're interested in theoretical research on alternatives to the current paradigm.

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#38

Post by sillysaurusx »

PartiallyTyped wrote: post #19

I recommend against DL by Goodfellow. At this point it is pretty much outdated. Actually, anything specific to NNs is already outdated by release. You'd need the following background: - Linear Algebra - Multivariate Calculus - Probability theory && Statistics Then you need a decent ML book to get the foundations of ML, you can't go wrong with either of these: - Bishop's Pattern Recognition - Murphy's Probabilistic ML…

(I can't wait until the myth that you need linear algebra and calculus to do ML finally dies. It's like saying that you need to understand assembly to do programming. It helps, but it's far from a requirement.)

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#40

Post by gaspb »

If you're more inclined to theory, I would suggest "Learning Theory from First Principles" by F. Bach: https://www.di.ens.fr/~fbach/ltfp_book.pdf

The book assumes limited knowledge (similar to what is required for Pattern Recognition I would say) and gives a good intuition on foundational principles of machine learning (bias/variance tradeoff) before delving to more recent research problems. Part I is great if you simply want to know what are the core tenets of learning theory!

Post reply on HN