I do like that there are lots of exercises.
Learning Theory from First Principles [pdf]
21–30 of 66 posts
Re: Learning Theory from First Principles [pdf]
#22Earlier quoted context omitted.
Ypur idea would become even better so if you read the book yourself ... You might even learn something new along the way.
we are optimizing for time here, not learning. I'm gonna die anyways and anything I learn will be dust. If I just need the info to make something work, spending months (of which I have limited number of) to see if something has something useful in it for me vs. spending and afternoon to probe it to get most of the benefits is a no-brainer.
Re: Learning Theory from First Principles [pdf]
#23This is pretty hard to read. For example, on the first page of chapter 1, it talks about "minimization of quadratic forms" and shows what looks like the formula for linear least squares. Is that right? It doesn't say anything about this. Some more exposition would help. I do like that there are lots of exercises.
Re: Learning Theory from First Principles [pdf]
#24I can’t wait until I tell GPT-5 “I have this idea I want to try, read this book and tell me there’s anything relevant there to make it work better”.
I've never heard of an LLM making up a new idea. Shouldn't this only work if your thing has already been tried before?
Re: Learning Theory from First Principles [pdf]
#25This is pretty hard to read. For example, on the first page of chapter 1, it talks about "minimization of quadratic forms" and shows what looks like the formula for linear least squares. Is that right? It doesn't say anything about this. Some more exposition would help. I do like that there are lots of exercises.
But your instincts are correct here. When you write out the objective function for ordinary least squares, it turns out to be a quadratic form. The choice of the word "quadratic" here is not a coincidence: it is the generalization of quadratic functions to matrices. That section covers the vector equivalent of minimizing quadratic functions.
Re: Learning Theory from First Principles [pdf]
#26This is pretty hard to read. For example, on the first page of chapter 1, it talks about "minimization of quadratic forms" and shows what looks like the formula for linear least squares. Is that right? It doesn't say anything about this. Some more exposition would help. I do like that there are lots of exercises.
I think the text is geared towards people with some mathematical background who want to understand learning theory. Besides it is clearly stated that this chapter is a review (so its assumed that you learned or will learn these things elsewhere).
This is older but is supposed to be good: https://www.deeplearningbook.org/
Re: Learning Theory from First Principles [pdf]
#27This is pretty hard to read. For example, on the first page of chapter 1, it talks about "minimization of quadratic forms" and shows what looks like the formula for linear least squares. Is that right? It doesn't say anything about this. Some more exposition would help. I do like that there are lots of exercises.
Re: Learning Theory from First Principles [pdf]
#28Interesting! I’ll have to look over it when I have more time. From a quick glance, it looks like it covers much of the same material as this text [1]. I wonder how they compare. [1]: https://www.cambridge.org/core/books/understanding-machine-l...
A 2014 book on machine learning sounds quaint and historical.
Re: Learning Theory from First Principles [pdf]
#29There are so many great mathematical PDFs available for free on the Internet, written by academics/educators/engineers. A problem is that there is a huge amount of overlap. I wonder if an AI model could be developed that would do a really good job of synthesizing an overlapping collection into a coherent single PDF without duplication.
Re: Learning Theory from First Principles [pdf]
#30This is pretty hard to read. For example, on the first page of chapter 1, it talks about "minimization of quadratic forms" and shows what looks like the formula for linear least squares. Is that right? It doesn't say anything about this. Some more exposition would help. I do like that there are lots of exercises.
Quadratic means square terms.