Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What a Deep Neural Network thinks about selfies

karpathy.github.io

Post reply on HN
Print view View original

31–40 of 56 posts

Re: What a Deep Neural Network thinks about selfies

#31

Post by nightpool »

the8472 wrote: post #27
nightpool wrote: post #16

>Be female. Women are consistently ranked higher than men. In particular, notice that there is not a single guy in the top 100. This sounds true, but it can't be the real reason—selfies are ranked relative to the other images by the same user . So unless users are taking a lot of #selfies of people of different genders, we can assume the dataset is already controlled for the gender of the person in the image, no? Unl…

Well, one of the other factors is long hair and the tendency to oversaturate the face. Those factors don't seem independent to me, men are less likely to sport long hair and they're also less likely to oversature the face to measure up to some skin perfection standards (think of it as the photographic equivalent of makeup). > but it can't be the real reason Can't? Ontop of the above-listed aspects it is entirely poss…

Neither of those explain why it would rank above the average of other female faces, in general.

Discussion about this with the author reveals that I was misinterpreting how they were collecting averages. I was assuming the "like" count was coming from each photo collected, but instead they collected the photos and average likes in individual steps, where the average likes were across recent posts by that user, rather then the selfies by that user.

Re: What a Deep Neural Network thinks about selfies

#33

Post by pshc »

Jack000 wrote: post #23
vonnik wrote: post #11

I think it's less about the head getting chopped than about having "the head take up about 1/3 of the image," as Karpathy says. So what the net is learning is composition, or balance in an image, which is really cool. The rule of thirds is actually pretty well know to people in photography: https://en.wikipedia.org/wiki/Rule_of_thirds (Our deep-learning framework http://deeplearning4j.org missed his list, but it's go…

possibly, but none of the cropped examples have cropped chins. It's also well known in photography that you can cut off someone's forehead, but never their chin.

Echoing a law of video games: "nobody looks up"!

Re: What a Deep Neural Network thinks about selfies

#34

Post by mirimir »

goodJobWalrus wrote: post #8
trhway wrote: post #3

looking at the top 100 one can only wonder how Hollywood has figured it out well before mighty power of computer :)

For me, this thing about having the top of your head cut from the picture is new. Who would have thought..

It seems that eyes and mouth, and their alignment, matter most for female attractiveness.[0,1]

[0] http://www.nbcnews.com/id/34482178/ns/health-skin_and_beauty...

[1] http://www.ncbi.nlm.nih.gov/pubmed/25836007

Re: What a Deep Neural Network thinks about selfies

#35

Post by karpathy »

nightpool wrote: post #31
the8472 wrote: post #27

Earlier quoted context omitted.

Well, one of the other factors is long hair and the tendency to oversaturate the face. Those factors don't seem independent to me, men are less likely to sport long hair and they're also less likely to oversature the face to measure up to some skin perfection standards (think of it as the photographic equivalent of makeup). > but it can't be the real reason Can't? Ontop of the above-listed aspects it is entirely poss…

Neither of those explain why it would rank above the average of other female faces , in general. Discussion about this with the author reveals that I was misinterpreting how they were collecting averages. I was assuming the "like" count was coming from each photo collected, but instead they collected the photos and average likes in individual steps, where the average likes were across recent posts by that user, rathe…

I screwed up on this point by the way - I had done this part of the experiment a few months ago and I incorrectly remembered the details. I went back and looked through the code and adjusted the post with more regarding this important point. In particular:

"Now it is time to decide which ones of those selfies are good or bad. Intuitively, we want to calculate a proxy for how many people have seen the selfie, and then look at the number of likes as a function of the audience size. I took all the users and sorted them by their number of followers. I gave a small bonus for each additional tag on the image, assuming that extra tags bring more eyes. Then I marched down this sorted list in groups of 100, and sorted those 100 selfies based on their number of likes. I only used selfies that were online for more than a month to ensure a near-stable like count. I took the top 50 selfies and assigned them as positive selfies, and I took the bottom 50 and assigned those to negatives. We therefore end up with a binary split of the data into two halves, where we tried to normalize by the number of people who have probably seen each selfie. In this process I also filtered people with too few followers or too many followers, and also people who used too many tags on the image."

Re: What a Deep Neural Network thinks about selfies

#36

Post by apu »

JabavuAdams wrote: post #32

How to take a good selfie: don't be black or dark-skinned, unless you're a celebrity. How do we prevent our AIs from learning racism? EDIT> Informative article, BTW. A good read.

This is an important point. People are thinking about it, and a lot of it will have to do with how the input data is gathered and curated.

Re: What a Deep Neural Network thinks about selfies

#37

Post by nl »

netheril96 wrote: post #14

One caveat with these machine inspired knowledge: they are prone to error, probably more than humans, at least for now. For example, if you train a CNN directly with human faces, its recognition rate comes way below what a human is capable of. Only after you apply tons of handcrafted optimizations, which are mostly black art, will you get close to or surpass a human's capability. Without much domain specific tuning,…

This is more wrong than right.

The example is correct, but not for the reasons stated. Humans are very, very good at face recognition. However, CNNs are pretty close to human performance for face detection.

Only after you apply tons of handcrafted optimizations, which are mostly black art, will you get close to or surpass a human's capability. Without much domain specific tuning, an AI's insight is far from reliable.

This just isn't the case. Take the GoogLeNet or VGGNet papers, build the CNN as described using Caffe/whatever, train as described in the paper and you'll end up with something that is pretty much on par with human performance for categorizing ImageNet images.

Take that same CNN architecture, and retrain it for another domain and it will perform roughly as well there too, for the task of categorizing into ~1K-10K image classes.

This isn't domain specific tuning. It's domain specific training, which is very different (although collecting the data is a big job).

Only after you apply tons of handcrafted optimizations, which are mostly black art, will you get close to or surpass a human's capability.

For CNNs, this is pretty much entirely false.

Re: What a Deep Neural Network thinks about selfies

#39

Post by yoha »

Tyr42 wrote: post #25
anunderachiever wrote: post #6

I would like to see a deep dream selfie ... Feed it an initial picture (noise, clouds, a selfie) and then backwards manipulate the input to maximize the assessed quality of the "selfie". I guess that would look pretty funny.

He did run something like that for cropping. He showed his favourite two "rude" ones at the bottom, where the 'Net cropped out the face of the person taking the selfie.

Actually, he used random crops and selected the highest rated. A "deep dream selfie" would actually run the neural network in reverse so as to generate a completely different image.

Re: What a Deep Neural Network thinks about selfies

#40

Post by visarga »

danblick wrote: post #2

This is neat. I bet Facebook or OkCupid are sitting on all sorts of click data that could be used to develop tools for helping people make their photos look better. (Even if, personally, I can't wait for a cultural backlash against internet narcissism...) [Edit: Even better, he didn't use click data to train the model, just public likes.]

The idea to use a convnet to reframe the selfie is neat. Makes it 5% better. Also, if it can be run on the phone, it could possible warn people they are about to post a shitty selfie before they do.
Post reply on HN