The early History of the Singular Value Decomposition (1993) [pdf]

Post by tzury » Sun, Jul 12, 2026, 3:19 AM UTC

jacobolus wrote: post #51

tzury wrote: post #48

Earlier quoted context omitted.

From the preface “You cannot read mathematics the way you read a novel. If you zip through a page in less than an hour, you are probably going too fast. When you encounter the phrase “as you should verify”, you should indeed do the verification, which will usually require some writing on your part. When steps are left out, you need to supply the missing pieces. You should ponder and internalize each definition. For e…

That's all well and good, but you also shouldn't excuse authors for not providing context, motivation, or explanation under the theory that the students should really be figuring out the whole subject from scratch for themselves. If you are really lucky the result of not explaining things might be an occasional exceptional student who works out a correct personal concept. But more commonly the result is just an unfil…

Post by jmalicki » Sun, Jul 12, 2026, 5:47 AM UTC

amarcheschi wrote: post #41

jmalicki wrote: post #15

Some fun stuff about SVDs: If you want to take a low rank approximation to a matrix D, let's call our approximation D'. The approximation that minimizes mean square error of the reconstructed matrix vs. the original (i.e. ||D - D'||_F, the Frobenius norm of their differences) happens to be the truncated SVD, by the Eckart–Young–Mirsky theorem [0]. I'm not claiming it's a practical way to do so, but this means that if…

I've recently passed my deep and generative learning exam and in fact before you said vae something was resonating with me, although in our course it wasn't made explicit we were approximating an svd. Staying in the field of autoencoder, it blows my mind how you can pass from denoising autoencoder (computer science) to scores and eventually matching flows (physics) quite seamlessly

Post by toolslive » Sun, Jul 12, 2026, 9:28 AM UTC

Aardwolf wrote: post #65

toolslive wrote: post #4

Earlier quoted context omitted.

Just going to sound really pedantic here, but RGB does not capture the entire colour space. In fact, it only captures about 35% of the colours the human eye can perceive. https://www.oceanopticsbook.info/view/photometry-and-visibil...

The eyes receptors are RGB too and they definitely capture all visible colors by definition (to be pedantic, yes there's also the rods for night vision, and the 'R' is more like yellow)

Post by amarcheschi » Sun, Jul 12, 2026, 11:00 AM UTC

jmalicki wrote: post #45

jmalicki wrote: post #15

Some fun stuff about SVDs: If you want to take a low rank approximation to a matrix D, let's call our approximation D'. The approximation that minimizes mean square error of the reconstructed matrix vs. the original (i.e. ||D - D'||_F, the Frobenius norm of their differences) happens to be the truncated SVD, by the Eckart–Young–Mirsky theorem [0]. I'm not claiming it's a practical way to do so, but this means that if…

I made a demo of the SVD part. https://jmalicki.github.io/svd-grad/ The devil was in the details, and that escalated quickly from a simple idea to actually getting it to work sucked me into a ton of random deep corners.

Post by FabHK » Sun, Jul 12, 2026, 12:57 PM UTC

FabHK wrote: post #43

BTW, if you wonder about the dedication ("For Gene Golub on his 15th birthday"): Gene Golub was a numerical analyst, and father of the practical singular value decomposition (his license plate read "Prof SVD"), together with William Kahan (the father of IEEE 754 floating point numbers). And his birthday was February 29. (In other words, the article was on the occasion of Gene's 60th birthday). https://en.wikipedia.or…