Live data from Hacker News

PixelNN – Example-Based Image Synthesis

cs.cmu.edu

111–120 of 155 posts

Re: PixelNN – Example-Based Image Synthesis

#111

"Enhance" is real. When will this stuff trickle into lower level law enforcement?

Hopefully never. This does not enhance the image - it makes up a plausible imaginary image.

EDIT: Furthermore the range of plausible imaginary images that match a given input is high (infinite?).

Re: PixelNN – Example-Based Image Synthesis

#112
post #84

It almost looks like they mixed training and testing data in some of the examples. The bottom-left sample in the normals-to-faces is extremely suspicions.

I was looking at this as well, but I'm willing to suspend my disbelief because the normal vaguely looks like it has a good deal of information (in a basic fidelity sense).

seems astonishing that the normal information includes enough detail to tell you which direction the eyes are pointing, though?

Re: PixelNN – Example-Based Image Synthesis

#113
post #7

So is there an analagous process that would apply to audio I wonder?

Yes, this is a fairly similar concept: https://magenta.tensorflow.org/nsynth

This is actually training a neural network on the Markov model, so it's very similar to core ideas behind the OP's paper. The core idea is to model the probability of a bit of sound by breaking it into the last note and everything that comes before the last note ("P(audio)=P(audio∣note)P(note)"). If you sample a bunch of audio and factor it that way for any given point in time, and accumulate that data somewhere, you can then sample the accumulated data randomly to generate new music.

There are other audio NN synthesis methods as well, pretty sure I've even seen one posted to ShowHN before.

Re: PixelNN – Example-Based Image Synthesis

#114

Earlier quoted context omitted.

Except, and this is really the fundamental catch, it's not so much "enhance" as it is "project a believable substitute/interpretation". You fundamentally can't get back information that has been destroyed/or never captured in the first place. What you can do is fill in the gaps/information with plausible values. I don't know whether this sounds like I'm splitting hairs, but it's really important that the general publ…

> You fundamentally can't get back information that has been destroyed/or never captured in the first place. I love this cliché. I've seen it thousands of times, and probably written it myself a few times. We all repeat stuff like that ad nauseam, without ever thinking. Because it's fundamentally flawed, especially in the context that it has usually been applied to, namely criticising the CSI:XYZ trope of "enhancing…

>> You fundamentally can't get back information that has been destroyed/or never captured in the first place.

> I love this cliché. I've seen it thousands of times, and probably written it myself a few times. We all repeat stuff like that ad nauseam, without ever thinking.

It is not a cliche it is an absolute truth. Information not present cannot be retrieved. There may be more information present than is immediately obvious.

> Neural Nets are excellent in squeezing all the information out noisy data

Maybe but they are also good at overfitting onto noisy data (the original article is an example of such overfitting).

Re: PixelNN – Example-Based Image Synthesis

#115

"Enhance" is real. When will this stuff trickle into lower level law enforcement?

Hopefully never. This does not enhance the image - it makes up a plausible imaginary image. EDIT: Furthermore the range of plausible imaginary images that match a given input is high (infinite?).

Just need to look at the picture of Fred Armisen to see that this technique can generate a picture of a plausibly real human who bears no/very little resemblance to the original image.

Re: PixelNN – Example-Based Image Synthesis

#117

I don't understand how the edges-to-faces can possibly work. The inputs seem to be black & white, and yet the output pictures have light skin tones. How can their algorithm work out the skin tone from a colourless image. Perhaps their training data only had white people in it?

You never saw edges2cats I take it? https://affinelayer.com/pixsrv/

> I don't understand how the edges-to-faces can possibly work. The inputs seem to be black & white, and yet the output pictures have light skin tones.

The step you're missing is that an edge detector is run on the entire database of training images to produce a database of edge images. The input edge image is run against that corpus of edge images in order to find which edge images match, then sample the corresponding original color images and synthesize a new color image.

Re: PixelNN – Example-Based Image Synthesis

#118

Earlier quoted context omitted.

Approaches like these are hallucinating the high resolution images though--not something that we'd ever want being used for police work. That said, I wonder if it would perform better than eyewitness testimony...

You could e.g. ostensibly produce valid license plates, which could be further reduced by matching the car color and model, to produce a small set of calid records.

You could also just pick a random license plate. It would be just as accurate.

Re: PixelNN – Example-Based Image Synthesis

#120
post #3

I used to roll my eyes at crime television shows, whenever they said "Enhance" for a low quality image. Now it seems the possibility of that becoming realistic are increasing with a steady clip, based on this paper and other enhancement techniques I've seen posted here.

This image from the article shows that the original image and the fantasy image are not alike at all. The faces look to have different ages. The computer even fantasized a beauty mark.

http://www.cs.cmu.edu/~aayushb/pixelNN/freq_analysis.png

The computer is fantasizing.

Post reply on HN