Principal Component Analysis Explained Visually (2015)

Nice visualization! This provides me an opportunity to go on a random tangent on PCA:

The post considers PCA from visualization perspective, but the exactly same thing can also be viewed as a method for reducing number of dimensions in the original dataset. [1] Now, one of the interesting questions in a dimensionality reduction task is, how to pick the number of dimensions (principal components)? A good number? In a principled way, instead of just computing the next component and the next and the one after that, until you get bored? (It works for visualizations where you often want only the first two or three components anyway, but suppose we want more information than plots.)

I recently learned that there's a fascinating way to do this, presented in Bishop's paper [2] from 1999. In short: this can answered by recasting the PCA as Bayesian latent variable model with a hierarchical prior. (Yes, it is a bit of mouthful to say. Yes, it is fairly mathematical, unlike the visualization.)

[1] https://en.wikipedia.org/wiki/Dimensionality_reduction

[2] https://www.microsoft.com/en-us/research/publication/bayesia...

I always used Jackson's broken-stick method for determining the number of dimensions to retain.

http://onlinelibrary.wiley.com/doi/10.2307/1939574/abstract

Previously at https://news.ycombinator.com/item?id=9040266.

Hello, HN. Co-author, here. Surprised to see this pop up again!

You can find the source code here: https://github.com/vicapow/explained-visually

Wish I had the free time to work on these more.

As someone who knows nothing about machine learning and nothing about PCA (well, until now :)), can someone please explain how the two relate to each other? Is one of them a subset to the other, or what?