Live data from Hacker News

DeepFace: Closing the Gap to Human-Level Performance in Face Verification

facebook.com

31–40 of 46 posts

Re: DeepFace: Closing the Gap to Human-Level Performance in Face Verification

#31
post #18

Earlier quoted context omitted.

This can already be done by someone who truly wants to keep an eye on you, only it takes more manpower (well, detective power). This is simply the same trend as ATMs replacing (some) bank tellers many years ago. Will it be easier to perform and abuse mass surveillance? Sure. Will people with something important to hide still wear disguises? I'd bet yes. My stance is that we can't fight progress, but we can start figh…

>Will people with something important to hide still wear disguises? I'd bet yes. Facial recognition systems can be (and are) trained to not be fooled by things like facial hair, hair changing, eyeglasses and sunglasses, hats, etc. Although it can obscure your features for sure. If someone does something really odd to try to avoid the facial recognition (I've people posting things like making your hair non-symmetrical…

And slowly, suspicious behavior is suppressed.

This will have two effects.

People who are truly suspicious for nefarious reasons will gradually stop being so.

On the other hand, many will be incorrectly flagged as suspicious and harassed without reason.

The trick is to strike a fair balance between the two. But who decides what's fair?

Re: DeepFace: Closing the Gap to Human-Level Performance in Face Verification

#32
post #13

Earlier quoted context omitted.

Actually no. For one thing, this isn't exactly a stealthy or cheap thing to do. It involves datacenters full of computing resources even for 4000 identies. I also don't believe it's so much a privacy issue. If I upload pictures to facebook I actually want them to be seen by human beings. The face-recognition only helps with that. If facebook recognizes me on a picture someplace else, I actually rather want to know ab…

Privacy issue is not about what you want, it's about what can be done (and often is without you knowing). Automated facial recognition is a serious privacy concern, it's not just about the slimy despicable thing facebook is. For example, in the uk where there are more cctv video feed than people to watch them, an automated facial recognition can track you around constantly. Their current automatic number plate recogn…

Wouldn't it actually be better to stop a government from doing that? If you can't, there's something wrong that no restrictions/bans on technological progresss will fix...

Usually it's easier to get the government in line than to prevent some technology to be developed...

Re: DeepFace: Closing the Gap to Human-Level Performance in Face Verification

#33
post #9

Important to note: - they still need 1000 labeled samples per identity - their network can only handle 4000 distinct identities (at 97.25% accuracy) at a time It's still a very worrying development for online and offline privacy.

Actually no. For one thing, this isn't exactly a stealthy or cheap thing to do. It involves datacenters full of computing resources even for 4000 identies. I also don't believe it's so much a privacy issue. If I upload pictures to facebook I actually want them to be seen by human beings. The face-recognition only helps with that. If facebook recognizes me on a picture someplace else, I actually rather want to know ab…

> If facebook recognizes me on a picture someplace else, I actually rather want to know about it. I'm not super famous, so unexpected pictures of myself can be more of a bad thing...

If all they did was notify you, that would be OK, I guess, although I'm not totally cool with it. I'm more worried about them seeing my picture and tagging it as me without my permission.

I think the crux of it is that facebook is pretty much the least trustworthy, most give-an-inch-they-take-a-mile company out there as far as privacy is concerned...

Re: DeepFace: Closing the Gap to Human-Level Performance in Face Verification

#34

Earlier quoted context omitted.

Actually no. For one thing, this isn't exactly a stealthy or cheap thing to do. It involves datacenters full of computing resources even for 4000 identies. I also don't believe it's so much a privacy issue. If I upload pictures to facebook I actually want them to be seen by human beings. The face-recognition only helps with that. If facebook recognizes me on a picture someplace else, I actually rather want to know ab…

> For one thing, this isn't exactly a stealthy or cheap thing to do. That's actually the more disturbing part about this: it require computational resources that only governments and large corporations can afford, so they are the ones who gain the most from it; and it gives them more leverage over the population.

"Leverage" is kind of vague. It's hard to make money from large-scale abuses...

And it doesn't look like the NSA for example is doing anything useful from its own perspective with the data they actually have...

Re: DeepFace: Closing the Gap to Human-Level Performance in Face Verification

#35
A paper posted on arxiv a few days ago by Fan et al. claims to have a similar level of accuracy (97.3% on LFW): http://arxiv.org/pdf/1403.2802.pdf

Both methods use deep neural networks, but have a lot of differences, e.g., the Fan et al. paper doesn't use a 3D face model.

Re: DeepFace: Closing the Gap to Human-Level Performance in Face Verification

#36
Having worked on this problem before (the comparison to human performance they cite is from my work) and seeing all the recent successes of deep learning, I'd bet that a lot of the gain here comes from what deep learning generally provides: being able to leverage huge amounts of outside data in a much higher-capacity learning model.

Let me try to break this down:

In machine learning, when you have input data that is labeled with the kinds of things you are directly trying to classify, that is called "supervised". In this case it's not quite supervised, because their main evaluations are on the LFW dataset, which is a verification dataset, whereas their training on SFC is a recognition task. The difference is that in verification, you are given photos of two people you've never seen before and have to identify if they're the same or not. In recognition, you are given one or more photos of several people as training data, and asked to identify a new face as one of them. In theory, you could build recognition out of verification (verify all pairs between training images and test input images and assign the top-scored name as the person) but in practice it's much better to build dedicated recognition classifiers for each person.

Their main network is trained on a recognition task, using their SFC dataset. They show these recognition results in Table 1 and the middle column of Table 2. An error number of 8.74% (DF-4.4M), for example, means that they were able to successfully name the person in 91.26% of input images. However, this error rate crucially depends upon two key factors: (1) the number of people they're trying to distinguish between, and (2) the number of images they have per person. For this test, it was ~4,000 people, and ~1,000 images/persons, respectively.

If you were to add more people to the database, or have fewer images per person, this accuracy would drop. You can see this clearly in Table 1, where subsets DF-3.3M and DF-1.5M have correspondingly lower error rates because they have fewer people (3,000 and 1,500, resp). Similarly, the middle column of that table shows how error rates rise when you reduce the number of images per person.

In contrast, all subsequent results are shown on verification benchmarks (LFW and Youtube Faces). In large part, I suspect this is because of the realities of publishing in the academic face recognition literature: you have to evaluate on some dataset that the community is familiar with to get your paper accepted, and LFW is the de-facto standard these days, and it only does verification not recognition.

Here, their performance is certainly very good, and an improvement over previous work, but not an unexpectedly huge leap. If you look at the LFW results page, you can see that recent papers have been edging up to this number quite steadily: 95.17% (high-dim LBP), 96.33% (TL Joint Bayesian), 97.25% (this paper) http://vis-www.cs.umass.edu/lfw/results.html

Nevertheless, how are they able to get this boost in performance? What recent papers in this field have increasingly been discovering is that having higher-dimensional features can really give you a big boost, or to put it another way: having a higher-capacity model is what buys you the additional performance.

In machine learning, the "capacity" of a model refers (in a loose sense) to how powerful it is. The basic tradeoff is that a higher-capacity learner can more accurately classify testing data BUT it requires much more training data to learn. The problem is that for the LFW benchmark, the amount of direct training data you have is strictly limited: there are 6,000 pairs of faces, and you train on 90% of them and test on the remaining 10%. This is not nearly enough data to train a high-capacity model.

So what people have been doing is training the bulk of their models on some other data, for some other task, and then adapt that model to the LFW problem, using the LFW training data essentially to "tweak" the classification model for this particular task. That's why the LFW results tables are now broken up into different sections according to how much outside data was used and in what form.

In the case of DeepFace, this takes the form of the SFC dataset and learning a network for recognition, not verification. Since they have access to lots of data of this form, they can successfully train a high-capacity model for it. Then they simply "chop-off" the last layer of the network -- the one that does the final recognition task, and instead replace it with a component for verification using only LFW training data. Or for their "unsupervised" results, using no LFW training data ("unsupervised" in quotes because it's not really unsupervised).

BTW, this approach of training a deep network for some task, and then cutting off the last layer to apply it to a different task (in effect making it simply a feature-extraction method) is quite common, and has been applied successfully to many problems that might not have enough data to train a high-capacity model directly.

Anyway, if people have more questions, I can try and answer them. (I'm not one of the authors, but I am in the field.)

Re: DeepFace: Closing the Gap to Human-Level Performance in Face Verification

#37
post #17

The key innovation is an accurate, reliable method for rotating faces so they're 'looking straight at the camera' before feeding them to a deep neural network. They call this 3D photo rotation process "frontalization." Figure 1 on page 2 of the paper shows at a very high level how this is being done. Very nice!

Actually, that's only one of the contributions, and I'm not so sure it's the "key innovation". Every other recent face recognition method also tries to do some kind of alignment to make faces more similar in pose/expression/lighting prior to classifying them; and of these, several also fit faces to a 3-d model to rotate to frontal (with varying quality).

See my other comment for my guess on what's actually providing the boost: https://news.ycombinator.com/item?id=7393378

Re: DeepFace: Closing the Gap to Human-Level Performance in Face Verification

#38

Earlier quoted context omitted.

I think you guys are reading different parts of the article. They present both results, supervised and unsupervised (where unsupervised uses the SFC dataset to train). They achieved 95.92% accuracy LFW with unsupervised (section 5.3) - so they can train on SFC and then classify a single image in a different domain with 95.92% accuracy. They achieved the 97.25% accuracy level was achieved as you say, when they let the…

I looked hard, and could not prove they can identify a single image with that accuracy. I couldn't tell WHAT the trial protocol was. Where is that in the document?

The LFW test is verification: given two faces you've never seen before, tell whether they're the same person or not. That's a much weaker test than saying "who is this?"

The LFW benchmark protocol is described in the original tech report: http://vis-www.cs.umass.edu/lfw/lfw.pdf

Re: DeepFace: Closing the Gap to Human-Level Performance in Face Verification

#39

Reducing error by 25% from a 96.5% baseline gives you their stated 97.25% accuracy. About 0.75% fewer errors. Still amazing, but less impressive than the abstract makes it sound.

Actually, 25% is the right way to judge this improvement. For example, let's say performance was currently at 99.9% and you improve it to 99.99%. That's not a 0.09% improvement (99.99 - 99.9), but rather a ten-fold improvement (.01% errors vs 0.1% errors).

This has to do with the fact that accuracy/errors are not linear.

Re: DeepFace: Closing the Gap to Human-Level Performance in Face Verification

#40
post #13

Earlier quoted context omitted.

Privacy issue is not about what you want, it's about what can be done (and often is without you knowing). Automated facial recognition is a serious privacy concern, it's not just about the slimy despicable thing facebook is. For example, in the uk where there are more cctv video feed than people to watch them, an automated facial recognition can track you around constantly. Their current automatic number plate recogn…

Wouldn't it actually be better to stop a government from doing that? If you can't, there's something wrong that no restrictions/bans on technological progresss will fix... Usually it's easier to get the government in line than to prevent some technology to be developed...

So how is that NSA reform coming along? Good I hope?
Post reply on HN