Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What are the foundational texts for learning about AI/ML/NN?

news.ycombinator.com

Post reply on HN
Print view View original

101–110 of 114 posts

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#101

Post by ranc1d »

jay3ss wrote: post #96
kevinskii wrote: post #35

Earlier quoted context omitted.

I agree. I read the first edition to Intro to Statistical Learning and it went into just the right level of mathematical depth. The authors also have Youtube lectures that accompany the chapters, and these are a great reinforcement of the material.

Do you have a link to the YouTube lectures? I'm taking a course and this is one of the books that we're using

It's linked off Trevor Hasties webpage see here:

https://www.youtube.com/playlist?list=PLoROMvodv4rOzrYsAxzQy...

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#102

Post by jgrimm »

Learning From Data (https://amlbook.com) is a great introduction to ML from a more theoretical perspective. The language is easy to understand but the concepts that it deals with are very theoretical, a combination that is hard to find elsewhere.

For example nearly everyone understands how to apply multivariable logistic regression, in say Numpy, however a good grasp of underlying concepts such as confidence bounds for overfitting and and being able to use formal proofs to explain concepts such as VC Generalisation will both help you stand out and provide a good foundation that makes further learning much easier.

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#103

Post by PartiallyTyped »

PartiallyTyped wrote: post #71
sillysaurusx wrote: post #57

Earlier quoted context omitted.

We agree, I think! And certainly, if you're one of those people who can pull it off, studying ML from first principles is probably an advantage. I just wince every time since I wouldn't have gotten into ML in the first place if I had to start with a big Calculus tome. There are probably a lot of people like me out there.

OP asked for foundational, and I provided _foundational_. In my opinion, everyone should start from some sound foundations in LinAlg and Calculus. Here are a couple of errors that stem from a single foundational problem: - a linear regressor can not be more than the number of datapoints - dimensionality reduction when you have NxM with M > N is bogus and you need a bigger dataset to do anything meaningful other than…

Correction:

A linear regressor can not have more parameters than the number of data points.

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#104

Post by robg »

rramadass wrote: post #97
robg wrote: post #43

Coming from cognitive neuroscience surprised that Explorations in Parallel Distributed Processing by McClelland and Rumelhart doesn’t get more attention as a classic in bridging old school AI approaches with the modern paradigm. https://psycnet.apa.org/record/1988-97441-000

This is nice; I am more interested in first understanding the origins/concepts/ideas behind AI/ML than in all the complicated mechanisms involved in implementing them (i.e. the simplest possible explanation/implementation) and hence these sort of books really interest me. Any more recommendations? PS: You might find Vehicles: Experiments in Synthetic Psychology by Valentino Braitenberg interesting if you don't alread…

I was raised on this era - you could go down quite the rabbit hole branching out from McClelland and Geoff Hinton. They were trying to reflect the brain and so backprop was initially seen as a shortcut that could be done away with once more complex models could be supported by processing power and inputs.

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#105

Post by nephanth »

raz32dust wrote: post #3

I personally consider Linear algebra to be foundational in AI/ML. Intro to Linear algebra, Gilbert Strang. And his free course on MIT OCW is fantastic too. While having strong mathematical foundation is useful, I think developing intuition is even more important. For this, I recommend Andrew Ng's coursera courses first before you dive too deep.

Linear algebra, and differential calculus (needs linear algebra), and a bit of optimisation (at least get an understanding of sgd)

Also proba/statistics! Without those you can end up doing stuff pretty wrong

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#107

Post by AadiBugg »

viscanti wrote: post #34
raz32dust wrote: post #3

I personally consider Linear algebra to be foundational in AI/ML. Intro to Linear algebra, Gilbert Strang. And his free course on MIT OCW is fantastic too. While having strong mathematical foundation is useful, I think developing intuition is even more important. For this, I recommend Andrew Ng's coursera courses first before you dive too deep.

Strang is great but he covers a lot of things that don't have much carryover to AI/ML and doesn't really cover things like Jacobians which do. Maybe there's something more useful for someone who is only learning Calculus and Linear Algebra for AI/ML than what Strang teaches.

These two O'Reilly books might be exactly what someone like this needs: https://www.oreilly.com/library/view/practical-linear-algebr... https://www.oreilly.com/library/view/essential-math-for/9781...

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#108

Post by nephanth »

jimbokun wrote: post #66
rg111 wrote: post #62

Earlier quoted context omitted.

You are plain exaggerating. You can't do all of them in a few weeks. Algorithms: Lin Reg -> Log Reg -> NN -> CNN + RNN -> GANs + Transformers -> ViT -> Multimodal AI + LLMs + Diffusion + Auto Encoders SVM, PCA, kNN, k-means clustering, etc. LightGBM, XGboost, Catboost, etc. Optimization and optimizers. Application-wise: Classification, Semantic Segmentation, Pose Estimation, Text Generation, Summarization, NER, Image…

> SVM, PCA, kNN, k-means clustering Are these still relevant in the age of Deep Neural Networks?

Also boosting.

But yes these algs are the basis of a lot of more modern algorithms.

A deep NN won't do unsupervised clustering for ex, and NNs perform more poorly than simpler models on small datasets

Re: Ask HN: What are the foundational texts for learning about AI/ML/NN?

#109

Post by nephanth »

popinman322 wrote: post #70
jimbokun wrote: post #66

Earlier quoted context omitted.

> SVM, PCA, kNN, k-means clustering Are these still relevant in the age of Deep Neural Networks?

PCA is a foundational dimension reduction technique, and kNN can be used in conjunction with embeddings. k-means is still great when you have prior/domain knowledge about the number of groups.

K-means is pretty poor when the clusters are not linearly separated, but it is the basis of a lot of more modern clustering techniques (kernel K-means if you have prior knoweledge, spectral clustering...)
Post reply on HN