I wonder if this could be applied to "incomplete" 3D models and the work shifted to the GPU!?
PixelNN – Example-Based Image Synthesis
71–80 of 155 posts
Re: PixelNN – Example-Based Image Synthesis
#72I hope some day this will generalize to video. I don't care about the exact shape of background trees in an action movie - with this approach, they could be compressed to just a few bytes, regardless of resolution.
Except that it can put trees somewhere where there were no trees but something similar to them. Or it can put face of a more popular actor instead of an actual less popular one because it was more often present in the training dataset. No, thanks.
Re: PixelNN – Example-Based Image Synthesis
#73Earlier quoted context omitted.
Except, and this is really the fundamental catch, it's not so much "enhance" as it is "project a believable substitute/interpretation". You fundamentally can't get back information that has been destroyed/or never captured in the first place. What you can do is fill in the gaps/information with plausible values. I don't know whether this sounds like I'm splitting hairs, but it's really important that the general publ…
On the other hand, this is what the brain does all the time.
Wouldn't it be ironic if a mystified and superstitious GAI emerges out of all these efforts.
Re: PixelNN – Example-Based Image Synthesis
#74OMG, now the "enhance" they say in investigative TV series and movies will actually be reality! :p
Re: PixelNN – Example-Based Image Synthesis
#75I don't understand how the edges-to-faces can possibly work. The inputs seem to be black & white, and yet the output pictures have light skin tones.
How can their algorithm work out the skin tone from a colourless image. Perhaps their training data only had white people in it?
Re: PixelNN – Example-Based Image Synthesis
#76Earlier quoted context omitted.
Except, and this is really the fundamental catch, it's not so much "enhance" as it is "project a believable substitute/interpretation". You fundamentally can't get back information that has been destroyed/or never captured in the first place. What you can do is fill in the gaps/information with plausible values. I don't know whether this sounds like I'm splitting hairs, but it's really important that the general publ…
> You fundamentally can't get back information that has been destroyed/or never captured in the first place. I love this cliché. I've seen it thousands of times, and probably written it myself a few times. We all repeat stuff like that ad nauseam, without ever thinking. Because it's fundamentally flawed, especially in the context that it has usually been applied to, namely criticising the CSI:XYZ trope of "enhancing…
It's not cliché, it's true. You fundamentally can't get back information that has been destroyed/or never captured in the first place.
Yes, a low-res image has lots of information. You can process that information in many ways. Missing data can't just be magically blinked into existence though.
Copy/pasting bits of guessed data is NOT getting back information that has been destroyed or never captured. Obscured data is very different from non-existent data. Could the software recreate a destroyed painting of mine based on a simple sketch? Of course not, because it would have to invent details it knows nothing about.
I think it's almost dangerous to call this line of thinking cliché. It should be celebrated, not ridiculed.
Re: PixelNN – Example-Based Image Synthesis
#77I don't understand how the edges-to-faces can possibly work. The inputs seem to be black & white, and yet the output pictures have light skin tones. How can their algorithm work out the skin tone from a colourless image. Perhaps their training data only had white people in it?
I had the same thought. Maybe it's not that there were only white people in the dataset, but it's actually taking the shape of the face into account, and it most closely matches those with white skin tones. I suggest this by looking at the cat one: it has the stripes coming off the eyes, so suggests one of the grey striped breeds rather than, e.g. all black or calico. It's probably more than pixel-by-pixel NN interpolation, but also taking into account some of the actual structure of the edges.
Re: PixelNN – Example-Based Image Synthesis
#78I don't understand how the edges-to-faces can possibly work. The inputs seem to be black & white, and yet the output pictures have light skin tones. How can their algorithm work out the skin tone from a colourless image. Perhaps their training data only had white people in it?
Color comes from the initial neural network step. Since skin color is relatively predictable from facial features (ex: nose width), it should be able to do reasonably well.
Re: PixelNN – Example-Based Image Synthesis
#79Earlier quoted context omitted.
Except, and this is really the fundamental catch, it's not so much "enhance" as it is "project a believable substitute/interpretation". You fundamentally can't get back information that has been destroyed/or never captured in the first place. What you can do is fill in the gaps/information with plausible values. I don't know whether this sounds like I'm splitting hairs, but it's really important that the general publ…
> You fundamentally can't get back information that has been destroyed/or never captured in the first place. I love this cliché. I've seen it thousands of times, and probably written it myself a few times. We all repeat stuff like that ad nauseam, without ever thinking. Because it's fundamentally flawed, especially in the context that it has usually been applied to, namely criticising the CSI:XYZ trope of "enhancing…
> It's similar to that guy who used Photoshop's swirl effect to hide his face, not knowing that the effect is deterministic, and can easily be undone.
The effect does not only need to be deterministic, but also invertible.
A low-res image has multiple "inverses" (yikes), supposedly each with an associated probability (if you would model it that way). So it would be more honest if the algorithm shows them all.
Re: PixelNN – Example-Based Image Synthesis
#80Earlier quoted context omitted.
Approaches like these are hallucinating the high resolution images though--not something that we'd ever want being used for police work. That said, I wonder if it would perform better than eyewitness testimony...
> hallucinating the high resolution images though To play devil's advocate though, modern neuroscience and neuropsychology basically tells us that that our brains reconstruct and recreate our memories every time we try to remember them. Our memories are highly malleable and prone to false implantation... and yet witness testimony is still the gold standard in courts.
And experts have been calling for a long time to at least limit the power of witness testimony, precisely for those reasons.