Some fun stuff about SVDs:
If you want to take a low rank approximation to a matrix D, let's call our approximation D'. The approximation that minimizes mean square error of the reconstructed matrix vs. the original (i.e. ||D - D'||_F, the Frobenius norm of their differences) happens to be the truncated SVD, by the Eckart–Young–Mirsky theorem [0].
I'm not claiming it's a practical way to do so, but this means that if you set up a neural network w/o nonlinearities that goes U -> S -> V^T, where S is a truncated scaling vector, and U and V^T are trained weights, make your loss function the MSE of reconstruction error, and minimize it with gradient descent, you will end up with the same U, S, and V that an SVD gives you.
In fact, this is basically exactly what a Variational Autoencoder [1] is! Way too few people realize this connection, and I wish it was taught in more ML courses. VAEs just add nonlinearities between U -> nonlinearity -> S -> nonlinearity -> V^T, and a KL-divergence regularization term. (Well VAEs are trained as operators to reconstruct vectors, and the S is an embedding not a trained weight, so I'm being a little sloppy, but still the connection is strong).
Once you realize this, you can have a lot of fun... anywhere you see an SVD being useful, you can construct arbitrary neural networks to replace them, and any time an SVD doesn't quite fit, e.g. you have binary data, realize that VAEs are just the same thing you can make all kinds of bespoke changes to... don't want MSE as your reconstruction error? Fine, use something else, but it's basically just an SVD!
[0] https://en.wikipedia.org/wiki/Low-rank_approximation#Basic_l...
[1] https://en.wikipedia.org/wiki/Variational_autoencoder