Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

3D Face Reconstruction from a Single Image

cs.nott.ac.uk

Post reply on HN
Print view View original

41–47 of 47 posts

Re: 3D Face Reconstruction from a Single Image

#41

Post by Hydraulix989 »

amelius wrote: post #38
andreyk wrote: post #8

Note that this is just one recent work in a well established research problem that has been worked on for decades. The interesting bit here is that they appear to achieve state of the art results by doing something simpler than other approaches - instead of fitting a well thought out generic morphable face model, they use a pretty standard deep learning model to map a 2d image to a discretized 3d model of the face (v…

I guess this means that the method easily generalizes to any type of object (?) > Intuitively this feels a bit off, since outputting a discretized 3d model instead of fitting a continuous model has inherent resolution limitations, but the benchmark results are pretty impressive. Isn't this because deep learning is in fact a type of interpolation? And another question: are there any techniques based on neural networks…

> Isn't this because deep learning is in fact a type of interpolation?

No The output is still discrete.

> And another question: are there any techniques based on neural networks that combine multiple images into the most plausible model?

Could work.

Re: 3D Face Reconstruction from a Single Image

#42

Post by Hydraulix989 »

exodust wrote: post #34

It struggles to create a likeness on the side profile but that's to be expected. Nose and mouth details lost and it ends up looking like someone else. Still it's very cool, but it can't perform miracles. A good test is when you have a front and side source image, such as police mug shot... Take this famous David Bowie mug shot. I also tried Jimi Hendrix and Jim Morrison, the side profile never looks like the person.…

You're never going to get an accurate side profile from ONE photo (front); otherwise, you are pulling information out of thin air -- like "enhancing" an image by somehow zooming in 100x.

Re: 3D Face Reconstruction from a Single Image

#43

Post by exodust »

Hydraulix989 wrote: post #42
exodust wrote: post #34

It struggles to create a likeness on the side profile but that's to be expected. Nose and mouth details lost and it ends up looking like someone else. Still it's very cool, but it can't perform miracles. A good test is when you have a front and side source image, such as police mug shot... Take this famous David Bowie mug shot. I also tried Jimi Hendrix and Jim Morrison, the side profile never looks like the person.…

You're never going to get an accurate side profile from ONE photo (front); otherwise, you are pulling information out of thin air -- like "enhancing" an image by somehow zooming in 100x.

Never say never.

A smarter algorithm might analyse the light and shadow to a greater degree, and better predict side profile details.

Or, take it further and have the algorithm silently check the internet for match of person and then look for more images of that person. Not cheating if the measure is "upload at least one image to start with, the service will then do the best it can to perform a miracle".

Re: 3D Face Reconstruction from a Single Image

#45

Post by Hydraulix989 »

exodust wrote: post #43
Hydraulix989 wrote: post #42

Earlier quoted context omitted.

You're never going to get an accurate side profile from ONE photo (front); otherwise, you are pulling information out of thin air -- like "enhancing" an image by somehow zooming in 100x.

Never say never. A smarter algorithm might analyse the light and shadow to a greater degree, and better predict side profile details. Or, take it further and have the algorithm silently check the internet for match of person and then look for more images of that person. Not cheating if the measure is "upload at least one image to start with, the service will then do the best it can to perform a miracle".

You're missing the point; if the information content isn't there (i.e. "no clues") and you're not providing it somehow (e.g. by "cheating" and providing more than one image from the Internet), there's literally no way to reconstruct it -- this reeks of the Nyquist sampling theorem.

It's not something that can still be accomplished by someone who isn't a pessimist; it's provably impossible.

Re: 3D Face Reconstruction from a Single Image

#46

Post by doublecorrect »

Hydraulix989 wrote: post #45
exodust wrote: post #43

Earlier quoted context omitted.

Never say never. A smarter algorithm might analyse the light and shadow to a greater degree, and better predict side profile details. Or, take it further and have the algorithm silently check the internet for match of person and then look for more images of that person. Not cheating if the measure is "upload at least one image to start with, the service will then do the best it can to perform a miracle".

You're missing the point; if the information content isn't there (i.e. "no clues") and you're not providing it somehow (e.g. by "cheating" and providing more than one image from the Internet), there's literally no way to reconstruct it -- this reeks of the Nyquist sampling theorem. It's not something that can still be accomplished by someone who isn't a pessimist; it's provably impossible.

That's, like, the entire opposite of the nyquist sampling theorem. Faces are a low dimensional space. Images are a higher dimensional space. https://en.m.wikipedia.org/wiki/Compressed_sensing

Re: 3D Face Reconstruction from a Single Image

#47

Post by Hydraulix989 »

doublecorrect wrote: post #46
Hydraulix989 wrote: post #45

Earlier quoted context omitted.

You're missing the point; if the information content isn't there (i.e. "no clues") and you're not providing it somehow (e.g. by "cheating" and providing more than one image from the Internet), there's literally no way to reconstruct it -- this reeks of the Nyquist sampling theorem. It's not something that can still be accomplished by someone who isn't a pessimist; it's provably impossible.

That's, like, the entire opposite of the nyquist sampling theorem. Faces are a low dimensional space. Images are a higher dimensional space. https://en.m.wikipedia.org/wiki/Compressed_sensing

There just isn't a learn-able (or "un-learnable" for that matter) function _even_ on the restricted domain of frontal profile images that maps surjectively onto the MUCH larger space of possible 3D face reconstructions. Intuitively, maybe the side of my jaw is deformed in a way that is not visible from the front, for example -- how can any oracle recover this information without seeing the side of my face?

Many people are mistakenly under the impression that certain signal reconstruction techniques like compressed sensing "violate" the Nyquist sampling theorem. Compressed sensing is still under the same umbrella of the Nyquist Sampling Theorem, as is CNN-based reconstruction (the technique used by this paper). I realize my analogy might have been poor; my claim is that there is still unrecoverable information loss.

Post reply on HN