Ask HN: How to learn AI from first principles? • Hacker News Board

Re: Ask HN: How to learn AI from first principles?

#31

Post by talles » Mon, Jan 27, 2025, 12:38 PM UTC

For deep learning:

1. Linear algebra. Be comfortable with vector transformations in the vector space. This is the framework to understand how data is represented and what is going on inside the model.

2. Calculus. Specifically derivatives, up to partial derivatives and the chain rule. This is needed to later understand backpropagation, the learning. It's fine to skip integrals.

3. Vanilla neural network. Study how a simple feed forward and fully connected neural network works, in detail. Every single bit about it.

I wouldn't worry or plan anything ahead until having those. After number 3 you'll have different branches to follow and will be better equipped to pick a path.

Re: Ask HN: How to learn AI from first principles?

#32

Post by dartos » Mon, Jan 27, 2025, 1:05 PM UTC

uncomplexity_ wrote: post #21

riwsky wrote: post #19

1. Brush up on linear algebra (matrix and vector arithmetic, etc) 2. Draw the rest of the fucking owl

can confirm, worked for me. i am now applying to current yc batch.

Hell, all you need to do for YC is fork continue.dev and name your company after a fruit.

Re: Ask HN: How to learn AI from first principles?

#33

Post by Bjartr » Mon, Jan 27, 2025, 1:34 PM UTC

I've been a fan of The Little Learner for the first principles side of things. It builds up the theory from almost nothing, step by step. It's got a conversational style that may turn some off, but I quite enjoyed it.

https://www.thelittlelearner.com/

Re: Ask HN: How to learn AI from first principles?

#34

Post by cdicelico » Mon, Jan 27, 2025, 1:36 PM UTC

A loved AI: A Modern Approach—still the best all around textbook on AI imho. I'd only add Susanna Epp's Discrete Mathematics with Applications, and I'd just focus on methodically and thoroughly working through those two, personally. First, Discrete Mathematics, then AI: A Modern Approach. This is exactly what I did and it was a great experience, super helpful.

Re: Ask HN: How to learn AI from first principles?

#35

Post by markus_zhang » Mon, Jan 27, 2025, 2:37 PM UTC

IMO learning AI from first principles means a Math/CS Master from a reputable CS university, preferably top 20.

Re: Ask HN: How to learn AI from first principles?

#36

Post by crimsoneer » Mon, Jan 27, 2025, 4:07 PM UTC

Going to plug the thoroughly excellent fast.ai courses here

Re: Ask HN: How to learn AI from first principles?

#37

Post by grepLeigh » Mon, Jan 27, 2025, 4:14 PM UTC

As a learning exercise, I enjoyed Neural Networks From Scratch: https://nnfs.io/

There's also a world of statistics and machine learning outside of deep learning. I think the best way to get started on that end is an undergrad survey course like CS189: https://people.eecs.berkeley.edu/~jrs/189/

Re: Ask HN: How to learn AI from first principles?

#38

Post by anonzzzies » Mon, Jan 27, 2025, 4:58 PM UTC

Bjartr wrote: post #33

I've been a fan of The Little Learner for the first principles side of things. It builds up the theory from almost nothing, step by step. It's got a conversational style that may turn some off, but I quite enjoyed it. https://www.thelittlelearner.com/

seconded

Re: Ask HN: How to learn AI from first principles?

#39

Post by rcarr » Mon, Jan 27, 2025, 5:13 PM UTC

Heard nothing but good things about Andrej Karpathy's series on Youtube

Re: Ask HN: How to learn AI from first principles?

#40

Post by jmholla » Mon, Jan 27, 2025, 6:50 PM UTC

It isn't first principles, but I would recommend 3blue1brown's ongoing series about neural networks [0]. I think there's a benefit to seeing the high level overview helps understand the purpose of the pieces as your learning them; it can help with motivation. Or watching overviews like this after the fact it can help bridge connections theory may not elucidate.

[0]: https://www.3blue1brown.com/topics/neural-networks