Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What are the foundational texts for learning about AI/ML/NN?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 114 posts

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#41

Post by sillysaurusx »

TaupeRanger wrote: post #37

There are none anymore. We now know that throwing a bunch of bits into the linear algebra meat grinder gets you endless high quality art and decent linguistic functionality. The architecture of these systems takes maybe a week to deeply understand, or maybe a month for a beginner. That's really it. Everything else is obsolete or no longer applicable unless you're interested in theoretical research on alternatives to…

A month to deeply understand?

I've been doing it since early 2019 and there are still subtleties that catch me off guard. Get back to me when you're not surprised that you can get rid of biases from many layers without harming training.

I broadly agree with you, but the timeline was just a little too aggressive. By about 10x. :)

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#43

Post by robg »

Coming from cognitive neuroscience surprised that Explorations in Parallel Distributed Processing by McClelland and Rumelhart doesn’t get more attention as a classic in bridging old school AI approaches with the modern paradigm.

https://psycnet.apa.org/record/1988-97441-000

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#44

Post by topspin »

sillysaurusx wrote: post #41
TaupeRanger wrote: post #37

There are none anymore. We now know that throwing a bunch of bits into the linear algebra meat grinder gets you endless high quality art and decent linguistic functionality. The architecture of these systems takes maybe a week to deeply understand, or maybe a month for a beginner. That's really it. Everything else is obsolete or no longer applicable unless you're interested in theoretical research on alternatives to…

A month to deeply understand? I've been doing it since early 2019 and there are still subtleties that catch me off guard. Get back to me when you're not surprised that you can get rid of biases from many layers without harming training. I broadly agree with you, but the timeline was just a little too aggressive. By about 10x. :)

> I've been doing it since early 2019 and there are still subtleties that catch me off guard.

That's true of every non-trivial discipline. I often learn subtleties about programming languages and hobbies I've been dealing with for decades.

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#45

Post by zffr »

You may also want to consider reading through some of the important (or highly cited) academic papers in AI/ML/NN. From these papers you may get a sense of the techniques researchers are using, and which topics are most important to learn.

I have not applied this technique to AI/ML/NN specifically, but it has been useful for me when trying to learn other topics.

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#46

Post by mythhouse »

NationalPark wrote: post #28
mfrieswyk wrote: post #7

Earlier quoted context omitted.

I never took beyond Precalculus in school, thanks for the tip!

Many of the suggestions so far are assuming you have taken undergraduate linear algebra and calculus. I'd start with those two subjects, you really can't build a foundational understanding of modern AI techniques without them.

i did linear algebra and calculus using strang and spivak textbooks. Those were classes i enjoy the most. But most of that stuff has atrophied from my brain over the years, do you recommend redoing those courses fast or can i learn when i need it on demand basis.

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#48

Post by cyber_kinetist »

TaupeRanger wrote: post #37

There are none anymore. We now know that throwing a bunch of bits into the linear algebra meat grinder gets you endless high quality art and decent linguistic functionality. The architecture of these systems takes maybe a week to deeply understand, or maybe a month for a beginner. That's really it. Everything else is obsolete or no longer applicable unless you're interested in theoretical research on alternatives to…

You still need to understand some basic theory/math about probabilistic inference (along with some knowledge of linear algebra), or else you’ll get a bit overwhelmed by some of the equations and not understand what the papers are talking about. PRML by Bishop is probably more than enough to start reading ML papers comfortably though. (This would probably be too easy for a competent math major, but not all of us are trained that way from the beginning…)

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#49

Post by dceddia »

I’m a big fan of learning through practice vs learning all the theory up front, and for anyone else who feels the same, the Fast AI course and book are very good: https://fast.ai

The authors are working on a new course that’ll dive deep into the modern Stable Diffusion stuff too, which I’m looking forward to.

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#50

Post by martythemaniak »

crosen99 wrote: post #5

"Neural Networks and Deep Learning", by Michael Nielsen http://neuralnetworksanddeeplearning.com (full text) The first chapter walks through a neural network that recognizes handwritten digits implemented in a little over 70 lines of Python and leaves you with a very satisfying basic understanding of how neural networks operate and how they are trained.

This is the thing that made NNs "click" for me, I think it was very good. Before this I did Andrew Ng's old ML course on coursera, so I thought that was a good intro to old ML approaches, common terms/techniques and flowed nicely into NNs.

But there's are both kinda old now, so there must be something newer that'll give you an equally good intro to transformers, etc.

Post reply on HN