Live data from Hacker News

Principal Component Analysis Explained Visually (2015)

setosa.io

21–28 of 28 posts

Re: Principal Component Analysis Explained Visually (2015)

#21
PCA can also help you study the relationships between variables.

I recently used it for a class project to explore the distribution of certain French cities in regards to socio-economic variables.

http://khalil.kacem.xyz:3838/#section-variables

You can see that Security and Economic Activity are opposite for example.

Re: Principal Component Analysis Explained Visually (2015)

#22
post #7

Earlier quoted context omitted.

you can look at it in terms of reconstruction error

Yeah PCA will give you eigenvalues of the PCs in descending order of variance explained so just summing and weighting those tells you the first 3 PCs explain say 93% of the variance in the data.

But the question is when exactly to stop, because the reconstruction error is always going to get lower. The bayesian solution, IIUC, is something like "stop when information required to store the new PCs is more than the information gained from the reduced reconstruction error."

The only issue with this is, if you get tons of data then there will be less uncertainty in the principal components. And so it will recommend as many as possible, even if they only decrease the reconstruction error a tiny bit.

Re: Principal Component Analysis Explained Visually (2015)

#24
post #10

As someone who knows nothing about machine learning and nothing about PCA (well, until now :)), can someone please explain how the two relate to each other? Is one of them a subset to the other, or what?

It is often used when you have an abundance of measured variables that you are using for input that you suspect might be highly correlated. For example in a study of self-reported lifestyle behaviors you might have questions about frequency of participation in: jogging, walking, running, weight lifting, cycling, aerobics classes, yoga, Crossfit, martial arts, climbing, tennis, softball, volleyball, golf, Ultimate Frisbee, and many more.

In your effort to predict whether a person will follow dietary guidelines for healthy eating you could just assign each activity as its own input to the model. Or, you could apply PCA (and something like varimax factor rotation) and what you might find is that these activities seem to reflect three somewhat separable latent variables that is: physical fitness, competitive athletics and friendship/team based social activity dimension. You now potentially have reduced 50 individual activity measures into 3 dimensions.

Next you would think more deeply about the specific items and combine them in into 3 scales and use the scales as a reduced dimensional input into the predictive model.

Re: Principal Component Analysis Explained Visually (2015)

#25
post #8

Hello, HN. Co-author, here. Surprised to see this pop up again! You can find the source code here: https://github.com/vicapow/explained-visually Wish I had the free time to work on these more.

I really love these visual explanations, and wish I had more at degree level. Do you think these will ever permeate traditional higher education?

Re: Principal Component Analysis Explained Visually (2015)

#26
post #16
post #7

Earlier quoted context omitted.

you can look at it in terms of reconstruction error

This is the standard way when looking for reduced order models in fluid mechanics. Variations on this: i) How 'faithfully' does it represent the data, eg, how many modes (components) are needed to resolve accuracy in a particular metric, or the entire system ii) What is the cut-off component number which has a signal of order of the measurement uncertainty.

He, I just happened to have MATLAB on my other screen performing a PCA of a CFD simulation. I got bored of watching the progress bar and decided to browse HN a bit. I guess I cannot escape...

Re: Principal Component Analysis Explained Visually (2015)

#27
post #2

Nice visualization! This provides me an opportunity to go on a random tangent on PCA: The post considers PCA from visualization perspective, but the exactly same thing can also be viewed as a method for reducing number of dimensions in the original dataset. [1] Now, one of the interesting questions in a dimensionality reduction task is, how to pick the number of dimensions (principal components)? A good number? In a…

Frank Harrell demonstrates a way to use PCA to reduce dimensionality with regression modeling strategies. His Course Notes (pdf) is a good reference point for multiple strategies on regression.

http://biostat.mc.vanderbilt.edu/wiki/Main/RmS

Post reply on HN