Imo, a better way to present this is to draw a diagram where various decompositions are connected by arrows that explain how one decomposition can be turned into another.
That is fascinating. Do you have an example of this which you can point to?
The big six matrix factorizations
41–50 of 84 posts
Re: The big six matrix factorizations
#42This is a fantastically clear outline of this topic. Thank you! > The terms “factorization” and “decomposition” are synonymous and it is a matter of convention which is used. Our list comprises three factorization and three decompositions. I can't tell if this is a joke: right after saying that these two words mean the same thing in this context, they are then used to categorize the methods. Edit: This is the kind of…
> I can't tell if this is a joke
It makes sense, the author is just using the original name of the method to bucket them e.g. spectral decomposition.
Re: The big six matrix factorizations
#43Any suggestions on what to learn in Linear Algebra after Gilbert Strang’s 18.06SC? https://ocw.mit.edu/courses/18-06sc-linear-algebra-fall-2011... My goal is to learn the math behind machine learning.
If you want a beautiful abstract perspective on linear algebra to complement Strang's more down-to-earth, matrix- and linear equation-oriented lectures, pick up Axler's Linear Algebra Done Right.
Re: The big six matrix factorizations
#44Thank you for this wonderfully concise summary: it’s convenient to have all this in one compact document. I suppose “flops” means “floating-point operations” here? Heretofore I’ve always encountered this as an abbreviation for “floating-point operations per second”.
When I used Matlab as an undergrad in the late 80's it used to report the flop count after each command, and those were floating point operations, no time units involved. I tried to find an image of the old command line but found this historical note first [1] and thought it would be of interest to readers of the article.
[1] http://www.stat.uchicago.edu/~lekheng/courses/309f14/flops/v... and
Re: The big six matrix factorizations
#45Spectral decomposition is pretty cool.
Application 1 - spectral clustering - an alternative to k-means for nonlinear clusters. Get a Distance matrix of your data, spectral decomp, run k-means on your k top eigen vectors and that's your clusters.
Application 2 - graph clustering - (run spectral clustering on adj matrix!)
There's some tricks to getting it to work in practice like normalizing but it's a simple and powerful method. Also the matrices can get big so it helps a lot to use sparse matrix libraries for the computations.
[1] https://towardsdatascience.com/spectral-clustering-aba2640c0....
Re: The big six matrix factorizations
#46Spectral decomposition is pretty cool. Application 1 - spectral clustering - an alternative to k-means for nonlinear clusters. Get a Distance matrix of your data, spectral decomp, run k-means on your k top eigen vectors and that's your clusters. Application 2 - graph clustering - (run spectral clustering on adj matrix!) There's some tricks to getting it to work in practice like normalizing but it's a simple and power…
I love that on this website I can find comments that I simply don't understand a word of. Good on you for doing the stuff you do sir.
Re: The big six matrix factorizations
#47This is a fantastically clear outline of this topic. Thank you! > The terms “factorization” and “decomposition” are synonymous and it is a matter of convention which is used. Our list comprises three factorization and three decompositions. I can't tell if this is a joke: right after saying that these two words mean the same thing in this context, they are then used to categorize the methods. Edit: This is the kind of…
the words are used synonymously in practice, but i think there's an opportunity to distinguish them usefully.
suggestion:
- 'factoring' is a multiplicative breakdown, a 'composition', like prime factorization.
- 'decomposition' could also be called 'partition', and is an additive breakdown, like how 3 could be split into 2 + 1
Re: The big six matrix factorizations
#48Any suggestions on what to learn in Linear Algebra after Gilbert Strang’s 18.06SC? https://ocw.mit.edu/courses/18-06sc-linear-algebra-fall-2011... My goal is to learn the math behind machine learning.
I am a big fan of Murphy's ML book [1].
[1] https://smile.amazon.com/Probabilistic-Machine-Learning-Intr...
It covers almost all the math you'd need to start doing ML research. I find it to be the ideal 'one book to rule them all' book for CS people in ML. Although, pure math grads in ML might find the book to not go deep enough.
Re: The big six matrix factorizations
#49Earlier quoted context omitted.
These lectures are fantastic after you've mastered the basics of linear algebra https://www.youtube.com/watch?v=McLq1hEq3UY (convex optimization, by a very experienced and often funny lecturer)
Stephen Boyd is a very good lecturer! I watched his videos on linear dynamical systems almost a decade ago and thought he did a fantastic job. Would highly recommend.
The problem set for the stanford linear dynamic systems that he taught is also fantastic. The level of difficulty and the breadth of applications leaves you with so many tools.
Re: The big six matrix factorizations
#50Missing the LDLT decomposition.