Live data from Hacker News

3D Face Reconstruction from a Single Image

cs.nott.ac.uk

21–30 of 47 posts

Re: 3D Face Reconstruction from a Single Image

#21
post #5

Earlier quoted context omitted.

Man are they good at marketing. They just knew this precise conversation would come up and created the perfect brief counter-argument preemptively ("hollywood quality masks").

Is it good marketing or engineering (the latter implies both)? I guess we'll find out soon enough.

If it were good only engineering no one would know about it

Re: 3D Face Reconstruction from a Single Image

#22
post #9
post #8

Note that this is just one recent work in a well established research problem that has been worked on for decades. The interesting bit here is that they appear to achieve state of the art results by doing something simpler than other approaches - instead of fitting a well thought out generic morphable face model, they use a pretty standard deep learning model to map a 2d image to a discretized 3d model of the face (v…

Maybe it is comparatively easy to fit the morphable model to the voxels?

This new method seems to produce nicer results when the input deviates from what would generic model expect (e.g. stronger facial expressions).

-----

Here is the same image used as input for 3d face reconstruction using both methods:

https://imgur.com/a/bZ012

Left side is morphable face model (EOS by Patrik Huber).

Right side is this new CNN voxels method.

-----

Here is live 3D model on the authors' site:

http://www.cs.nott.ac.uk/~psxasj/3dme/view.php?name=../59b41...

Re: 3D Face Reconstruction from a Single Image

#24
post #4

It definitely constructs a face, but how accurate is it? Would be interesting to see a measurement of error against a known result. Maybe try rendering an image of a 3d model of a face, then attempt to reconstruct it from the rendered image and measure the displacement from the original model.

Well, they include an error metric in the paper - basically average vertex distance of predicted face and actual face model (in interocular distance units). They seem to show that they are much better than state of the art with respect to that error, but of course if you want to interpret what that actually means then it's probably easier to interpret the error with your suggestion of actually visualizing it.

Re: 3D Face Reconstruction from a Single Image

#25
post #4

It definitely constructs a face, but how accurate is it? Would be interesting to see a measurement of error against a known result. Maybe try rendering an image of a 3d model of a face, then attempt to reconstruct it from the rendered image and measure the displacement from the original model.

It's fitting faces to some kind of assumed model. I tried it on a few Japanese faces; it basically gave them a Caucasian-looking side profile: big foreheads with over-hanging eyebrow ridges; big noses. Completely unrecognizable after turning more than a few degrees from front view. Which is not to flame the project---nice work!---but as a potential improvement for certain applications, it would be useful to simply ha…

You could use an expert system to classify the face by race. Then fit it over the appropriate model.

Re: 3D Face Reconstruction from a Single Image

#26

Earlier quoted context omitted.

If you look at the .obj model file that you can download you can see exactly how accurate it it. The mesh is dense and the detail is not. The detail that it does have however is very plausible.

One of the authors here. You are very right in saying that the there aren't many details. This limitation, we believe, is due to a lack of large, high quality training sets. The data we trained from was very smooth, which means our method is unable to pick out features such as wrinkles and dimples.

What about the "Maybe try rendering an image of a 3d model of a face, then attempt to reconstruct it from the rendered image and measure the displacement from the original model." approach for generating high resolution training data?

Re: 3D Face Reconstruction from a Single Image

#27
post #20
post #18

Earlier quoted context omitted.

> static 3d-printed would not work right, but adding in fake eyeballs should do the trick. For inspiration, check out how these guys fool Samsung's iris scanner: https://www.youtube.com/watch?v=4VrqufsHpS4

They mentioned an infrared camera, bet they're checking temperature too.

They're using a near infrared camera which can't see much beyond what your eye can see except for TV remote LEDs and security camera floodlights. It can't see temperature unless something is almost red hot.

Re: 3D Face Reconstruction from a Single Image

#28
post #20
post #18

Earlier quoted context omitted.

> static 3d-printed would not work right, but adding in fake eyeballs should do the trick. For inspiration, check out how these guys fool Samsung's iris scanner: https://www.youtube.com/watch?v=4VrqufsHpS4

They mentioned an infrared camera, bet they're checking temperature too.

should be able to warm up a mask with a blow dryer

Re: 3D Face Reconstruction from a Single Image

#30
post #26

Earlier quoted context omitted.

One of the authors here. You are very right in saying that the there aren't many details. This limitation, we believe, is due to a lack of large, high quality training sets. The data we trained from was very smooth, which means our method is unable to pick out features such as wrinkles and dimples.

What about the "Maybe try rendering an image of a 3d model of a face, then attempt to reconstruct it from the rendered image and measure the displacement from the original model." approach for generating high resolution training data?

That's actually a great idea. Because then you can get a diff between the outputted 3d and the actual 3d used to generate the images, as opposed to what I assume is just diffing the generated profile with a profile shot of the subject.
Post reply on HN