Looks like exactly what I was looking for :p
Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
91–100 of 177 posts
Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
#92I don't think the content of the comments in this thread is limited to ML. I think there is lot of applied math research out there (almost all of it?) that hardly anyone outside of academia actually reads.
I think there's some useful stuff but my impression is that research papers are mostly dead ends so I stick to graduate textbooks. Maybe other people have other approaches? I'm not a math researcher so I don't need to be at the cutting edge.
Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
#93I don't think the content of the comments in this thread is limited to ML. I think there is lot of applied math research out there (almost all of it?) that hardly anyone outside of academia actually reads. I think there's some useful stuff but my impression is that research papers are mostly dead ends so I stick to graduate textbooks. Maybe other people have other approaches? I'm not a math researcher so I don't need…
[deleted]
Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
#94Earlier quoted context omitted.
The last part was a big hurdle for me as an early undergrad. I was a fairly strong programmer toward the end of high school, and was trying to think of math as programming. That worked for the fairly algorithmic high school stuff and I got good grades, but it made I was awful at writing proofs. I also went through a phase where I used all the logical notation and rules to manipulate it possible in order to make proof…
This is essentially my problem. I started writing programs at a young age and was introduced (unknowingly) to many more advanced mathematical concepts from that perspective rather than through pure mathematics. What was it that helped break this paradigm for you?
Really trial and error and grinding through proofs. Working through Linear Algebra Done Right was a big a-ha moment for me. Since I was self-studying over the summer (to remedy my poor linear algebra skills), I was very rigorous in making sure I understood every line of the proofs in the text and trying to mimic his style in the exercises.
In hindsight, I think the issue was trying to map everything to programming is a bad idea and I was doing it because programming was the best tool in my tool chest. It was a real “when all you have is a hammer, everything looks like a nail” issue for me.
Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
#95Earlier quoted context omitted.
> "the scaling hypothesis" Oh boy, don't get me started.... I first off should say that by no means do I think any of these people (at least those publishing) are dumb. You can also be a genius in one direction and a fucking idiot in another, and that's okay. Certainly describes me haha (well less on the genius side and more on the functioning idiot side. So take everything I say with a grain of salt). Don't get me w…
Very interesting. Are any of your lectures available online?
I'm trying not to dox myself so I can be more open on HN (though more concerns in modern era...). You can find some harsh words against some ML community practices in my history and I think it is easy to get misinterpreted as calling people dumb or confuse academic bashing from utility (I criticize LLMs and diffusion a lot because I like them, not the other way). So yes and no. But the lectures I have aren't recorded and public (zoom for my Uni. I'm ABD in my PhD). My lecture slides and programs should be publicly visible though, but I don't go into this with them because I've been specifically asked to not teach this way :/ In all fairness, our ML course only has Calc 1 as a pre-req and CS students aren't required to take Lin Alg (most do though, but first courses are never really that great ime) or differential equations. TBH to get into this stuff you kinda need some metric theory. If you actually poke through this paper you'll find that come up very quickly, and this is common in the optimal transport community. But I think if you get into metric theory a lot of this will make sense pretty quickly. So if you can, maybe start with Shao's Mathematical Statistics?
Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
#96[flagged]
Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
#97As someone who has a deeper knowledge of programming rather than math, I find the mathematical notation here to be harder to understand than the code (even in a programming language I do not know). Does anyone with a stronger mathematical background here find it easier to understand the math as written more easily than the source code?
Use ChatGpt. Screenshot the math, crop it down to the equation, paste into the chat window. It can explain everything about it, what each symbol means, and how it applies to the subject. It’s an amazing accelerator for learning math. There’s no more getting stuck. I think it’s underrated because people hear “LLM’s aren’t good at math”. They are not good at certain kinds of problem solving (yet), but GPT4 is a fantast…
Don't suggest this. While I agree it can be helpful, the problem is if you're a novice you won't be able to distinguish hallucinations. Which in my experience are fairly common, especially as you do advance topice. If you got good math rigor then it's extremely helpful, because often things are hard to exactly search, but it's a potential trap for novices. But if you have no better resource, then I can't blame anyone, just give a warning to take care.
Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
#98Earlier quoted context omitted.
The last part was a big hurdle for me as an early undergrad. I was a fairly strong programmer toward the end of high school, and was trying to think of math as programming. That worked for the fairly algorithmic high school stuff and I got good grades, but it made I was awful at writing proofs. I also went through a phase where I used all the logical notation and rules to manipulate it possible in order to make proof…
I don't know that has anything to do with programming. Arithmatic and writing proofs are very different skills. There is going to be a gap for everyone.
Yeah I know it’s a common challenge. I think it took me a bit longer than some of my peers because I was trying to force it to be like something I knew instead of meeting it on its own terms.
When all you have is a hammer and all that
Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
#99As someone who has a deeper knowledge of programming rather than math, I find the mathematical notation here to be harder to understand than the code (even in a programming language I do not know). Does anyone with a stronger mathematical background here find it easier to understand the math as written more easily than the source code?
As someone that’s in the later stages of a PhD in math, given the title starts with “Mathematical Introduction…”, the notation feels pretty reasonable for someone with a background in math. Sure I might want some slight changes to the notation I found skimming through on my phone, but everything they define and the notation they choose feels pretty familiar and I understand why they did what they did. Mirroring what…
Is it fair to call something an introduction if it uses math from an upper division undergrad math criteria? Such as metric theory. My opinion is that it is context driven. E.g. Introduction to Differential Geometry or Introduction to Homotopy Theory. But I think you can't look at the title and infer prerequisites that are within the ballpark. I'd wager most people outside math and some physics students are familiar with Galerkin methods (maybe a handful of engineers) at the undergraduate level. I don't think most outside math and physics even learn PDEs (my engineering friends mostly didn't and my uni's CS program doesn't even require DE).
Re: Mathematical Introduction to Deep Learning: Methods, Implementations, and Theory
#100for those who want some maths-heavy stuff for deep learning, check francois fluret's book https://fleuret.org/francois/lbdl.html. the pdf is free but the print is so cute.