The big six matrix factorizations • Hacker News Board

Re: The big six matrix factorizations

#81

Post by edelman » Thu, May 19, 2022, 12:00 PM UTC

MatteoFrigo wrote: post #37

the_svd_doctor wrote: post #27

Earlier quoted context omitted.

It's always been very ambiguous in practice. To avoid ambiguity I usually use `flop/s` but not everyone likes that :)

Speaking of SVD doctors, I heard many years ago (from Alan Edelman) that Gene Golub's license plate used to be "DR SVD". Later he switched to "PROF SVD". I couldn't confirm the DR SVD part, but the PROF SVD story appears to be real: https://www.mathworks.com/company/newsletters/articles/profe...

As I recall the Dr SVD was the license on his second car. He once left the car at SFO airport and when he heard I was flying in, he asked me to drive it to his house (keys under the seat!) What he didn't tell me was that it was a "stick shift" and I only once before had a friend teach me how that works. After driving around San Francisco for Gene I got better at it, but it was definitely scary at times. :-)

Re: The big six matrix factorizations

#82

Post by mbrudd » Thu, May 19, 2022, 12:28 PM UTC

time_to_smile wrote: post #63

melling wrote: post #4

Any suggestions on what to learn in Linear Algebra after Gilbert Strang’s 18.06SC? https://ocw.mit.edu/courses/18-06sc-linear-algebra-fall-2011... My goal is to learn the math behind machine learning.

The vast majority of the mathematics required to really understand ML is just probability, calculus and basic linear algebra. If you know these already and still struggle it's because the notation is often terse and assumes a specific context. Regarding this the only answer is to keep reading key papers and work through the math, ideally in code as well. For most current gen deep learning there's not even that much m…

Amen!! This is the best way to learn anything technical -- put things into practice to understand the theory. It's also important to keep revisiting the theory to understand results, rather than parroting some catchphrase to "explain" results.

Re: The big six matrix factorizations

#83

Post by heinrichhartman » Thu, May 19, 2022, 12:37 PM UTC

whatshisface wrote: post #35

heinrichhartman wrote: post #32

Earlier quoted context omitted.

It's not unique, but it can be implemented in a way that is stable. E.g. SVD does this (but is quite overkill for the requirements). It's highly relevant from an algebraic perspective, hence it's curious that it's not covered (at all) in the numeric literature.

I could be misunderstanding, but I think you'd get all the algebraic content out of it by computing the dimensions of the image and kernel, then working with them from there on. Why would you want to have a matrix decomposition that separated them? Granted, computing the dimensions of the kernel is not so easy, especially because a pair of vectors can be arbitrarily close without being linearly dependent. No wonder t…

> I could be misunderstanding, but I think you'd get all the algebraic > content out of it by computing the dimensions of the image and kernel, > then working with them from there on. Why would you want to have a > matrix decomposition that separated them?

Well. Sometimes you want to know a solution to a problem and not only the dimension of the solution space : )

Also for composition: If you have "compatible" matrices B,C. How do you compute the restriction: A|_ker(B), A|_im(B) or co-restrictions (factor projections): A/ker(C), A/im(C), etc.

Re: The big six matrix factorizations

#84

Post by alpple » Tue, May 24, 2022, 8:09 PM UTC

throwawaymaths wrote: post #66

arbitrandomuser wrote: post #58

Earlier quoted context omitted.

I've never given a second thought about what the etymology of "spectral" in spectral decomposition is. Somewhere in the back of my mind (and I guess many students of physics have the same notion) subconsciously i assumed it originates from eigenvalues of the Hamiltonian determining the atomic spectral lines . But I've never followed up on it and actually looked it up .

According to the almighty wikipedia, The connection is correct but it turned out to be an accident. David Hilbert who coined spectral theory was surprised when it was found to be applicable to solving quantum mechanical spectra.

If a spectrum is just a range of numbers, then the idea of spectrum should apply to many phenomenon. Many things can be described by a number. So, when matrix spectrum and atomic spectrum were formulated, perhaps spectrum was a word that was in vogue to describe a quantity. I think this is the explanation of the reason the same term is used for both. Because many connected phenomenon can be quantified. I know only a little about matrix spectrum and atomic spectrum, so take my thought with a grain of salt.