Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An illustrated introduction to linear algebra

ducktyped.org

Post reply on HN
Print view View original

61–70 of 91 posts

Re: An illustrated introduction to linear algebra

#61

Post by PanoptesYC »

Miserlou57 wrote: post #38

about 15 years ago I started an aggregator to accumulate/sort/filter the best instruction of various topics, kinda like Reddit for learning. This is such a perfect example of the kind of thing I hoped would filter to the top. Thinking about trying to redo it. Is there a use for this sort of thing in today's world?

An easily searchable platform with curated high quality guides would be a good place to start when trying to do anything. Guides aren't something I'd want to stumble on, like YC posts, but something I would be seeking out. Probably a top feature would be a robust tagging system/search engine rather than the social Reddit elements like karma, hot page, trending subs, etc. Would be cool!

Re: An illustrated introduction to linear algebra

#62

Post by barrenko »

xwowsersx wrote: post #7

This is great. I really appreciate visual explanations and the way you build up the motivation. I'm using a few resources to learn linear algebra right now, including "The No Bullshit Guide to Linear Algebra", which has been pretty decent so far. Does anyone have other recommendations? I've found a lot of books to be too dense or academic for what I need. My goal is to develop a practical, working understanding I can…

Ok, boy, I'm also reviewing LinAlg textbooks as we speak. Coming in with a similar interest for ML / AI.

I've done math on KA academy up to linear algebra, with other resources / textbooks / et al. depending on the topic.

People will recommend 3B1B, Strang (MIT OCW Lin Alg lessons). For me the 3B1B is too "intuitionist" for a first serious pass, and Strang can be wonderful but then go off on a tangent during a lecture that I can't follow, it's a staple resource that I use alongside others.

LADR4e is also nice but I can't follow the proofs there sadly (yet). There is also 'Linear Algebra done wrong', as well as the Hefferon book, which all end up being proof-y quite quickly. They seem like they'll be good for a second / third pass at a linear algebra.

Side note - for a second or a third pass in LA it seems there is such a thing as 'abstract linear algebra' as a subject and the texbooks there don't seem that much harder to follow than the "basic" linear algebra ones designated for a second pass.

I've gotten off to the most of a start with ROB101 textbook (https://github.com/michiganrobotics/rob101/blob/main/Fall%20...), up until linear dependence / independence, along the MIT Strang lectures. ROB101 is nice as it deals with the coding aspect of it all, and I can follow in my head as I am used to the coding aspect of ML / AI.

I also have a couple obscure eastern european math texbook(s) for practice assignments.

Most lately I have been reviewing this course / book - https://www.math.ucdavis.edu/~linear/ (which has cool notes at https://www.math.ucdavis.edu/~linear/old), and getting a lot of mileage from https://math.berkeley.edu/~arash/54/notes/.

Re: An illustrated introduction to linear algebra

#63

Post by imtringued »

blackbear_ wrote: post #36
imtringued wrote: post #33

Earlier quoted context omitted.

Since you're associating ML with singular value decomposition, do you know if it is possible to factor the matrices of neural networks for fast inverse jacobian products? If this is possible, then optimizing through a neural network becomes roughly as cheap as doing half a dozen forward passes.

Not sure I am following; typical neural network training via stochastic gradient descent does not require Jacobian inversion. Less popular techniques like normalizing flows do need that but instead of SVD they directly design transformations that are easier to invert.

The idea is that you already have a trained model of the dynamics of a physical process and want to include it inside your quadratic programming based optimizer. The standard method is to linearize the problem by materializing the Jacobian. Then the Jacobian is inserted into the QP.

QPs are solved by finding the roots (aka zeroes) of the KKT conditions, basically finding points where the derivative is zero. This is done by solving a linear system of equations Ax=b. Warm starting QP solvers try to factorize the matrices in the QP formulation through LU decomposition or any other method. This works well if you have a linear model, but it doesn't if the model changes, because your factorization becomes obsolete.

Re: An illustrated introduction to linear algebra

#64

Post by great_wubwub »

This is clear and useful but I wish you'd picked different example numbers. Using 1 and 2 for both bread and milk makes it harder to look at the matrix form and immediately see whether a 1 in a matrix is the bread 1 or the milk 1. If you could use 1,2,3,4 instead of 1,2,1,2 it would make things much clearer.

Re: An illustrated introduction to linear algebra

#65

Post by egonschiele »

MehdiHK wrote: post #52
egonschiele wrote: post #14

Earlier quoted context omitted.

Thank you! I loved writing that book.

One of my favorite books! Any plans to make this series a book as well? (Will be an instant buy for me)

I'll definitely publish more chapters on substack! I'd love to do a full book on LA, but it depends on if I'll have time.

As an aside, Avro.im looks awesome!

Re: An illustrated introduction to linear algebra

#66

Post by egonschiele »

ngriffiths wrote: post #32

I feel like it's obligatory to also drop a link to the 3blue1brown series on linear algebra, for anyone interested in learning - it is a step up from what's in this post, but these videos are brilliant and still super accessible: https://youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFit...

3B1B's videos are incredible. His LA videos go a bit too quick for me though, which is why I started writing this series.

Re: An illustrated introduction to linear algebra

#67

Post by egonschiele »

rzz3 wrote: post #51
egonschiele wrote: post #13

Earlier quoted context omitted.

Author here – I think you're probably right. I wrote the Gaussian elimination section more as a recap, because I figured most readers have seen Gaussian elimination before, and I was keen to get to the rest of it. I'd love to hear if other folks had trouble with this section. Maybe I need to slow it down and explain it better.

I hadn’t, and your article lost me there to be honest. You didn’t explain the what, why, or when behind it, and it didn’t make sense to me at all. That said, I’m abnormally horrible at math.

Noted! I may make a totally separate post on gaussian elimination. Could you talk me through what parts were confusing, and would you be willing to review a post on gaussian elimination to see if it works for you?

Re: An illustrated introduction to linear algebra

#69

Post by moron4hire »

great_wubwub wrote: post #64

This is clear and useful but I wish you'd picked different example numbers. Using 1 and 2 for both bread and milk makes it harder to look at the matrix form and immediately see whether a 1 in a matrix is the bread 1 or the milk 1. If you could use 1,2,3,4 instead of 1,2,1,2 it would make things much clearer.

I agree with this critique because, with learning linear algebra, there are a lot of numbers flying around and the order of them is very important. This is why I like to use the prime sequence for my example numbers, because you can also see where they contributed to results of multiplication operations.
Post reply on HN