Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares

vmls-book.stanford.edu

Post reply on HN
Print view View original

71–80 of 126 posts

Re: Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares

#71

Post by Koshkin »

CamperBob2 wrote: post #52
RossBencina wrote: post #47

Earlier quoted context omitted.

> A (standard) unit vector... I know this as "an element of the standard basis," B = {e_1, e_2, ...), where e_1 = (1,0,0,...), e_2 = (0,1,0,0,...). You could view it as inauspicious that the treatment doesn't begin with abstract vector spaces, but there is always Axler. For what it's worth, I find it inauspicious that after taking three (pure-math oriented) Linear Algebra courses I never saw least squares nor the SVD…

Point being, their definition is just plain wrong. If that's how the authors describe a unit vector, I don't think this is the book you want to use to learn about SVD.

Well, the way I see it, when you teach applied science you often want to sacrifice some rigor so that your students could focus on what was intended to be learned in the first place.

Re: Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares

#72

Post by jcranmer »

k__ wrote: post #39

Somehow I found linear algebra easier than calculus, but I don't know why. I did both at the same time in university, but failed calculus 3 times and aced linear algebra at the first try. I'd expect being either good or bad at math, not both at the same time

> Somehow I found linear algebra easier than calculus, but I don't know why.

I suspect the answer is that your calculus course was a lot heavier on crank-grinding: having to readily apply integration and differentiation on a wide panoply of functions, some of them you're not really familiar with (such as arccos). If you're weak on trigonometry or some algebraic manipulations, that's going to shut out the ability to do a lot of the crank-grinding without really impacting your ability to understand the concepts.

By contrast, the crank-grinding in linear algebra is a lot less involved. The most complex algebra is going to be solving polynomial equations to find the eigenvalues of a matrix, but those are generally going to mostly be quadratic equations since asking anyone to solve more complex equations by hand is going to ask for trouble. Otherwise, it's largely plug-and-chug numbers into stock formula. Gram-Schmidt orthonormalization? Pick a vector, normalize it, project the other vectors and cancel them out, and repeat until you've done all of them.

Re: Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares

#73

Post by Connarhea »

muhneesh wrote: post #9
flor1s wrote: post #7

Earlier quoted context omitted.

Don't forget to review calculus as well. Khan Academy is a good start for learning about single variable calculus ( http://www.khanacademy.org ), but their content on multivariable calculus is a bit lacking (neural networks / deep learning use the concept of the derivatives and the gradient a lot). A good supplement for multivariable calculus would be Terence Parr and Jeremy Howard's article on "All the matrix calcul…

Thanks - I am doing that as well! I've been using MIT OpenCourseware for single variable calculus (and will do the same for multivariable). I fenced the parent post to Linear Algebra to not go too far away from the OP. I will certainly check out the Terrence Parr / Jeremy Howard site, and am super familiar with Khan Academy.

I'm coming to the end of my first year (6 year part time) Comp Sci course and have seen that we have options for AI and Machine Learning modules in future years. Where should I go to find something like a list of what I should be brushing up on, or learning completely from scratch, in order to not fall flat on my face during those type of modules.

I understand there are very set starting points in math subjects because concepts build on one another but I don't know what I should be starting with and where to go afterwards.

Re: Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares

#74

Post by jackallis »

muhneesh wrote: post #4

I'm e-learning Linear Algebra right now to have a good math foundation for Machine Learning. I was a History and Sociology major in college - so I didn't take any math. If you are like me, and working off an initial base of high school math, I would recommend the following (all free): Linear Algebra Foundations to Frontiers (UT Austin) Course: https://www.edx.org/course/linear-algebra-foundations-to-fro... Comments:…

did you find LAFF too mathy? i got turned off by the mathyness of it and i quit in 2 weeks. does it get any better? All the math notations and lines got so dry that i vapourised trying to understand.

Re: Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares

#75

Post by notyourloops »

hackermailman wrote: post #17
TobiasA wrote: post #11

I'm a self taught programmer with a very weak maths background. What's the best learning path for me if I want to be able to understand and create ML based applications?

There's a practical course for this http://www.datasciencecourse.org/lectures/ anything you don't know, like linear algebra, look up the topics here for a 1-2hr crash course https://www.youtube.com/playlist?list=PLm3J0oaFux3aafQm568bl... There's a playlist for a math background in ML for anybody who wants to try a more rigorous ML course https://www.youtube.com/playlist?list=PL7y-1rk2cCsA339crwXMW... More information…

> There's no answers because the author gives thanks to a grad course in evidenced based teaching where he claims the only way to really know something and remember it is to figure it out for yourself. Math stackexchange can help too.

This is a cop out; of course to really know something and remember you have to figure it out for yourself. But answers allow you to check whether your work was right, and if not, allow you the opportunity to debug your work.

My best performance came in organic chemistry, where I looked for question banks (with answer keys) and solved problems extensively, perhaps bordering on obsessively. If I hadn't an indicator that my final result was wrong, I would have missed out on many learning opportunities, and objectively my performance would have been worse. In general, I have found this strategy to enable me to be an exceptional student.

If you don't benefit from an answer key, you're probably lazy and undisciplined. Alternatively, you have too much time on your hands, opting to rigorously confirm that each and every answer is correct.

In short, by not providing an answer key, you are denying the disciplined student the opportunity to efficiently learn.

Re: Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares

#76

Post by cultus »

syntaxing wrote: post #45

Applied linear algebra is such a great idea. Linear algebra is relatively easy to understand and used everywhere. But the material is so damn boring since it's a lot of arthimetic. Even the homework problem is boring since there is no specific purpose.

Typical LA courses in math departments have a bizarre focus on being able to do Gaussian elimination by hand and stuff like that. It's not particularly useful or even mathematically interesting. LA courses would be so much more useful if they just stuck to theory and only had computer applications.

Re: Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares

#77

Post by graycat »

Koshkin wrote: post #60
graycat wrote: post #36

I went though the slides. Super fun material! I've seen all the methods long ago, and much deeper than in the slides, and published on some of the most advanced material, and much more, but, still, it was fun material because of the many examples and really good graphs. From their other books, clearly they are real experts. The slides, then, are a careful path where minimal theory gives a LOT of nice applications. Th…

> determinants... CAN be omitted Also see http://www.axler.net/DwD.html .

Looks like a nice paper!

The paper says how to go beyond what is in Boyd, et al., i.e., eigenvalues, eigenvectors, the spectral decomposition, etc. without determinants. Nice!

For that material I would have been tempted just to use the old approach of determinants and the roots of the characteristic polynomial, the Hamilton-Cayley theoem, etc.

Saved the paper! Thx.

Re: Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares

#79

Post by ivansavz »

muhneesh wrote: post #4

I'm e-learning Linear Algebra right now to have a good math foundation for Machine Learning. I was a History and Sociology major in college - so I didn't take any math. If you are like me, and working off an initial base of high school math, I would recommend the following (all free): Linear Algebra Foundations to Frontiers (UT Austin) Course: https://www.edx.org/course/linear-algebra-foundations-to-fro... Comments:…

That's a pretty good list, here are some things I'd add.

Amazing js visualizations/manipulatives for many LA concepts: http://immersivemath.com/ila/index.html

LA Concept map: https://minireference.com/static/tutorials/conceptmap.pdf#pa... (so you'll know what there is to learn)

Condensed 4-page tutorial: https://minireference.com/static/tutorials/linear_algebra_in... (in case you're short on time)

And here is an excerpt from my book: https://minireference.com/static/excerpts/noBSguide2LA_previ... (won't post a link to it here, but check on amazon if interested)

Re: Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares

#80

Post by muhneesh »

jackallis wrote: post #74
muhneesh wrote: post #4

I'm e-learning Linear Algebra right now to have a good math foundation for Machine Learning. I was a History and Sociology major in college - so I didn't take any math. If you are like me, and working off an initial base of high school math, I would recommend the following (all free): Linear Algebra Foundations to Frontiers (UT Austin) Course: https://www.edx.org/course/linear-algebra-foundations-to-fro... Comments:…

did you find LAFF too mathy? i got turned off by the mathyness of it and i quit in 2 weeks. does it get any better? All the math notations and lines got so dry that i vapourised trying to understand.

I didn't think about it during the time. It's a fair comment, and probably true.

What it did really well (for me) was integrate HW with each lecture video, and start at a really basic foundation. It took me from 0 -> something.

Post reply on HN