Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How to learn AI from first principles?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 63 posts

Re: Ask HN: How to learn AI from first principles?

#41

Post by jonnycoder »

andyjohnson0 wrote: post #30

Back in 2018 I did Andrew Ng's course in Machine Learning on Coursera. It was pretty much "from first principles" in that you learned a bit of linear algebra and then you implemented algorithms in Octave, working up to MNIST etc. I felt like I came out of it with a good understanding of the basics, and that ML is maths not magic. Looks like the course has turned into a multi-course "specialization" and I have no idea…

I've taken part 1 of 3 in Andrew Ng's machine learning specialization which covers the math for supervised learning, linear regression, etc. As I started part 2 (neural networks), it built off the math from part 1 such as the sigmoid activation function. This is what I think of when the OP refers to learning ML from first principles. I highly recommend Andrew Ng's course and I feel like I need to take it again to really understand those basic building blocks.

Re: Ask HN: How to learn AI from first principles?

#42

Post by gamblor956 »

Step 1, abandon the concept of "first principles." First principles don't exist for most areas of study outside of pure mathematics, and for newer fields like AI they haven't been established yet so you'd just be hamstringing yourself.

Step 2: The steps above are a good plan for learning about traditional AI, and the traditional approaches, which were based on an attempt to model human thought processes. Machine learning was what the industry turned to in the early 2000s because we didn't have the hardware capabilities then to meaningfully model neural networks. We do now, but machine learning has taken over so there's very little research into modeling neural networks...about the same as there was when I was an undergrad.

Re: Ask HN: How to learn AI from first principles?

#43

Post by HardikVala »

jmholla wrote: post #40

It isn't first principles, but I would recommend 3blue1brown's ongoing series about neural networks [0]. I think there's a benefit to seeing the high level overview helps understand the purpose of the pieces as your learning them; it can help with motivation. Or watching overviews like this after the fact it can help bridge connections theory may not elucidate. [0]: https://www.3blue1brown.com/topics/neural-networks

3blue1brown's content on NN's is awesome -- The explanations are super intuitive. But I'm also looking to understand, as you say, the big picture and understand where NN's fit.

Re: Ask HN: How to learn AI from first principles?

#44

Post by HardikVala »

Maro wrote: post #28

Hi, it depends on what you mean by "first principles". If you don't have a solid background in math, then that's what you should improve upon (calculus, linear algebra, discrete math, probability theory, information theory). Some of the books you mention do cover this at the beginning, but most people take separate courses on these topics at University, with lots of homework, etc. Also, the first book on your list is…

AIMA is wide-ranging, a lot of which is not "must-have", but "nice-to-have" knowledge. But I do like its breadth-over-depth approach to get a full scope of the AI landscape.

Re: Ask HN: How to learn AI from first principles?

#45

Post by HardikVala »

cdicelico wrote: post #34

A loved AI: A Modern Approach—still the best all around textbook on AI imho. I'd only add Susanna Epp's Discrete Mathematics with Applications, and I'd just focus on methodically and thoroughly working through those two, personally. First, Discrete Mathematics, then AI: A Modern Approach. This is exactly what I did and it was a great experience, super helpful.

Will check out Discrete Mathematics, thanks!

Re: Ask HN: How to learn AI from first principles?

#48

Post by HardikVala »

grepLeigh wrote: post #37

As a learning exercise, I enjoyed Neural Networks From Scratch: https://nnfs.io/ There's also a world of statistics and machine learning outside of deep learning. I think the best way to get started on that end is an undergrad survey course like CS189: https://people.eecs.berkeley.edu/~jrs/189/

Was not aware of these resources. Thanks for sharing!
Post reply on HN