On Eigenfaces: Creating ghost-like images from a set of faces

Isn't this one of the homework's in the stanford/coursera ml course? I feel like this is not really original content

Here's an animation of an autoencoder learning filter weights. It's interesting that they look similar.

https://lambdal.com/images/autoencoder-learning-face-filters...

It doesn't contain any new ideas, no - there are many other tutorials about eigenfaces with example code, such as:

http://jeremykun.com/2011/07/27/eigenfaces/

http://nbviewer.ipython.org/github/rcquan/sklearn-practice/b...

The wikipedia article (https://en.wikipedia.org/wiki/Eigenface) also contains code for a MATLAB implementation.

It's not completely by chance. There's an old paper [1] that shows that if the activation functions are well approximated using only up to the linear term of its Taylor expansion, then the optimal weights for encoding and decoding are the same as PCA.

There's probably newer results on this topic; I'm sure.

However, I will say that I've created some autoencoders on toy sets like those found in scikit-learn, and the spaces learned via the autoencoder and the spaces found through PCA were often similar if not identical. For example, if my input vectors were in R^n (with n > 3) and I restricted an autoencoder to 3 units, the encoding matrix of the autoencoder would span the same subspace as the first 3 principal component directions.

[1]: http://oucsace.cs.ohiou.edu/~razvan/courses/dl6900/papers/bo...

[Author here] Definitely never intended to claim that this was an original discovery; the original paper using the term is ~25 years old [http://www.cs.ucsb.edu/~mturk/Papers/mturk-CVPR91.pdf].

Nonetheless, I've found it to be an interesting concept. There is indeed a homework from the Coursera ML course for computing and visualizing eigenfaces, and the course (and the Stanford CS229 notes) discuss PCA further. I decided to explore the ideas further and distill it into a blog post specifically on eigenfaces.

Goal is for it to serve as a condensed tutorial on an interesting topic! I definitely learned a bunch writing it, and it may be interesting to others who have yet to come across to concept.