This is incredible. It seems to perform well on faces with dysmorphic features including cleft lips, asymmetric eyes, and abnormal locations of facial landmarks. It even creates an appropriate 3D representation of a cleft. I've found that some other face detection implementations actually fail at even localizing faces with these abnormalities. I don't have the background to fully understand the paper, though. Is there anything truly novel in how they are detecting faces? Could their work be leveraged to more accurately label facial landmarks for "abnormal" faces?
3D Face Reconstruction from a Single Image
31–40 of 47 posts
Re: 3D Face Reconstruction from a Single Image
#32This is incredible. It seems to perform well on faces with dysmorphic features including cleft lips, asymmetric eyes, and abnormal locations of facial landmarks. It even creates an appropriate 3D representation of a cleft. I've found that some other face detection implementations actually fail at even localizing faces with these abnormalities. I don't have the background to fully understand the paper, though. Is ther…
I'm really happy to hear that it works on those sorts of cases. I've tried a few images during testing, such as unusually long noses, and was pleased with how they came out. The novelty comes from a simple approach to a usually quite complex problem (i.e. posing the problem as a semantic segmentation problem, to produce a spatially aligned volume)
Re: 3D Face Reconstruction from a Single Image
#33Earlier quoted context omitted.
It's fitting faces to some kind of assumed model. I tried it on a few Japanese faces; it basically gave them a Caucasian-looking side profile: big foreheads with over-hanging eyebrow ridges; big noses. Completely unrecognizable after turning more than a few degrees from front view. Which is not to flame the project---nice work!---but as a potential improvement for certain applications, it would be useful to simply ha…
You could use an expert system to classify the face by race. Then fit it over the appropriate model.
That is the wrong tool for the wrong problem. You don't need to classify by race, and an expert system would be a horrible way to do it.
Just get more Asian people in the training data.
Re: 3D Face Reconstruction from a Single Image
#34It struggles to create a likeness on the side profile but that's to be expected. Nose and mouth details lost and it ends up looking like someone else. Still it's very cool, but it can't perform miracles.
A good test is when you have a front and side source image, such as police mug shot...
Take this famous David Bowie mug shot. I also tried Jimi Hendrix and Jim Morrison, the side profile never looks like the person.
https://ozimg.s3.amazonaws.com/temp/bowie_mug.jpg
An improvement to the engine might also accept a side profile source image to improve the details. Now that would be something!
Re: 3D Face Reconstruction from a Single Image
#35As far as the demo user interface goes, I find the controls to be weird. Dragging the 3D model left and right will move the nose left and right. But dragging up and down will move the face model in the opposite direction.
That is a bit inconsistent!
Try the right mouse button instead; it seems to have a nicer rotation action, and the two axes are consistent. Also you can use the middle mouse button to zoom in.
Re: 3D Face Reconstruction from a Single Image
#36Amazing.
It's also amazing how far Vision has come in a few short years.
Re: 3D Face Reconstruction from a Single Image
#37Tryied it on Harry Rowolth- didnt get along with the beard.
Re: 3D Face Reconstruction from a Single Image
#38Note that this is just one recent work in a well established research problem that has been worked on for decades. The interesting bit here is that they appear to achieve state of the art results by doing something simpler than other approaches - instead of fitting a well thought out generic morphable face model, they use a pretty standard deep learning model to map a 2d image to a discretized 3d model of the face (v…
I guess this means that the method easily generalizes to any type of object (?)
> Intuitively this feels a bit off, since outputting a discretized 3d model instead of fitting a continuous model has inherent resolution limitations, but the benchmark results are pretty impressive.
Isn't this because deep learning is in fact a type of interpolation?
And another question: are there any techniques based on neural networks that combine multiple images into the most plausible model?
Re: 3D Face Reconstruction from a Single Image
#39Earlier quoted context omitted.
They mentioned an infrared camera, bet they're checking temperature too.
should be able to warm up a mask with a blow dryer
You can detect a person's pulse by amplifying the right frequencies in a video, because the pressure of the heartbeat changes the color of the face almost simultaneously. You could probably fool that by giving the mask capillaries, but that would end up being expensive.
Re: 3D Face Reconstruction from a Single Image
#40Like all these machine learning generation things, it provides a plausible model, but not necessarily the correct model.