This article takes a journey from first principles: https://harishnarayanan.org/writing/artistic-style-transfer/
Ask HN: How to learn AI from first principles?
11–20 of 63 posts
Re: Ask HN: How to learn AI from first principles?
#12I’m starting down a similar path with these two books
How AI Works - https://nostarch.com/how-ai-works
+
Why Machines Learn: The Elegant Math Behind Modern AI - https://www.penguinrandomhouse.com/books/677608/why-machines...
Re: Ask HN: How to learn AI from first principles?
#13I recommend the following video lectures from a course: https://www.youtube.com/@deeplearningsystemscourse1116/
It covers ground I don't know where else to find on getting from theory to practice.
Re: Ask HN: How to learn AI from first principles?
#14The question depends what you mean by first principles. Usage of the phrase "first principles" has sprawled into many different things since (I think) Musk first mentioned it as a way to learn. The original, philosophical meaning of first principles meant a fundamental truth which could be used to derive others. Much of the philosophising of thinkers like Aristotle or Descartes was to uncover these truths (eg I think…
Ya, the phrase "first principles" is vague...I meant starting from an axiomatic and actionable definition of AI and learning from there. The first chapter of AIMA does a swell job of enumerating different definitions of and then explicitly declaring which one is used and the foundational premises for the concepts and methods to follow. And it doesn't define AI then jump to neural networks, it gradually layers more at…
By "first principles" do you mean something long "learn from the ground up" or " from basic building blocks"?
I like learning things starting from small, atomic this, then building up and learning higher layers of abstraction and functionality later. I tend to find hands on totals too "top down" in the sense that they start with all the told in place and then give you a cursory look into what's actually happening.
Personally I feel like most things in the world aren't really that complicated when you understand the building blocks. There are a few core ideas and then a bunch of layers on top to organize and utilize those ideas for different applications. So if I have an interest in something I want to learn from the ground up.
Re: Ask HN: How to learn AI from first principles?
#15I’m starting down a similar path with these two books How AI Works - https://nostarch.com/how-ai-works + Why Machines Learn: The Elegant Math Behind Modern AI - https://www.penguinrandomhouse.com/books/677608/why-machines...
Thanks for the recommendations. What's your impression of these texts so far?
Re: Ask HN: How to learn AI from first principles?
#16The question is rather a bit flawed without knowing your background, as to determine what to suggest. That being said, I would argue a lot of the novelty and recent advances in AI/ML are not yet documented in books, but rather in dense scientific papers.
Re: Ask HN: How to learn AI from first principles?
#17Watch Karpathy's 'Zero to Hero' videos on YouTube.
If you want a historical perspective, which is very worthwhile, start by reading about the mid-century work of McCullough and Pitts, and Minsky, Papert and their colleagues at MIT CSAIL after that.
There will be a dry spell after Minsky and Papert because of their conclusion that the OG neural-network topology that everyone was familiar with, the so-called "perceptron", was a dead end. That conclusion was premature to say the least, but in any event the hardware and training techniques weren't available to support any serious progress.
Adding hidden layers and nonlinear activation functions to the perceptron network seemed promising, in that they worked around some of Minsky's technical objections. The multi-layer perceptron was now a "universal approximator" capable of modeling any linear or nonlinear function. In retrospect that should have been considered a bigger deal than it was, but the MLP was still a PIA to train, and it didn't seem very useful at the scales achievable in hardware at the time. Anything a neural net could do, specialized code could usually do better and cheaper.
Then, in the circa-2010 timeframe, AlexNet dusted off some of the older ideas and used them to win image-recognition benchmark competitions, not by a small margin but by blowing everybody else into the weeds. That brought the multi-layer perceptron back into vogue, and almost everything that has happened since can be traced back to that work.
The Karpathy videos are the best intro to the MLP concept I've run across. Understanding the MLP is the key prereq if you want to understand current-gen AI from first principles.
Re: Ask HN: How to learn AI from first principles?
#18Earlier quoted context omitted.
Ya, the phrase "first principles" is vague...I meant starting from an axiomatic and actionable definition of AI and learning from there. The first chapter of AIMA does a swell job of enumerating different definitions of and then explicitly declaring which one is used and the foundational premises for the concepts and methods to follow. And it doesn't define AI then jump to neural networks, it gradually layers more at…
By "first principles" do you mean something long "learn from the ground up" or " from basic building blocks"? I like learning things starting from small, atomic this, then building up and learning higher layers of abstraction and functionality later. I tend to find hands on totals too "top down" in the sense that they start with all the told in place and then give you a cursory look into what's actually happening. Pe…
Exactly. Nailed it.
Re: Ask HN: How to learn AI from first principles?
#191. Brush up on linear algebra (matrix and vector arithmetic, etc)
2. Draw the rest of the fucking owl
Re: Ask HN: How to learn AI from first principles?
#20https://a16z.com/ai-canon/
I prefer the a16z AI canon for this purpose. It’s useful and historical. It’s structured to begin with no prerequisites and work up to cutting edge research papers. And best of all it’s free and open source.