Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PixelNN – Example-Based Image Synthesis

cs.cmu.edu

Post reply on HN
Print view View original

91–100 of 155 posts

Re: PixelNN – Example-Based Image Synthesis

#91

Post by tgb »

jampekka wrote: post #61
avian wrote: post #25

I found the title somewhat misleading. I was expecting some clever application of the nearest-neighbor interpolation. But this seems to involve neural nets and appears far from "simple" to me (I'm not in the image processing field though).

AFAIU it actually seems to be sort of "just" a clever application of the nearest-neighbor interpolation. The CNN is used to come up with the feature space for the pixels (weights of the CNN), and then each pixel is "copy-pasted" from the training set based on the nearest match. It seems that this could be used in theory with any feature descriptors, such as local color histograms, although the results wouldn't probab…

I think the confusion is that the term "nearest neighbor approach" has a different meaning in machine learning than in image interpolation.

https://en.wikipedia.org/wiki/Nearest-neighbor_interpolation

versus

https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm

Re: PixelNN – Example-Based Image Synthesis

#92

Post by gfredtech »

tgb wrote: post #91
jampekka wrote: post #61

Earlier quoted context omitted.

AFAIU it actually seems to be sort of "just" a clever application of the nearest-neighbor interpolation. The CNN is used to come up with the feature space for the pixels (weights of the CNN), and then each pixel is "copy-pasted" from the training set based on the nearest match. It seems that this could be used in theory with any feature descriptors, such as local color histograms, although the results wouldn't probab…

I think the confusion is that the term "nearest neighbor approach" has a different meaning in machine learning than in image interpolation. https://en.wikipedia.org/wiki/Nearest-neighbor_interpolation versus https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm

+1, the exact conclusion i came to(K-nearest neighbors) when I saw this post. Thanks for pointing this out

Re: PixelNN – Example-Based Image Synthesis

#93

Post by ZeroGravitas »

jlebrech wrote: post #33
otp124 wrote: post #3

I used to roll my eyes at crime television shows, whenever they said "Enhance" for a low quality image. Now it seems the possibility of that becoming realistic are increasing with a steady clip, based on this paper and other enhancement techniques I've seen posted here.

It can give possible matches, i don't think it would be admissible in court. they could still trick a confession out of someone using that image.

You don't specify, but presumably you mean a true confession.

It could also be used to generate a false confession. If the prosecutor says "We have proof you were there at the scene" and shows you some generated image, then you as an innocent person have to weigh the chances of the jury being fooled by the image (and even if it's not admissable in court, it may be enough to convince the investiging team that you are responsible and stop looking for the real perpetrator) and the expected sentences if you maintain your innocence vs "admitting" your guilt.

Re: PixelNN – Example-Based Image Synthesis

#94

Post by mlevental »

dannyw wrote: post #90
jokoon wrote: post #86

I have a large collection of images, many being accessible through google image search. I wonder if there could be a way to "index" those images so I can find them back without storing the whole image, using some type of clever image histogram or hashing-kind function. I wonder if that thing already exist, since there are many images, and since most images have a lot of difference in their data, could it be possible…

That's perceptual hashing. Check out https://www.phash.org/

so will this do something like image recognition? ie does it work as well as surf/sift?

Re: PixelNN – Example-Based Image Synthesis

#95

Post by WhitneyLand »

ACow_Adonis wrote: post #4
otp124 wrote: post #3

I used to roll my eyes at crime television shows, whenever they said "Enhance" for a low quality image. Now it seems the possibility of that becoming realistic are increasing with a steady clip, based on this paper and other enhancement techniques I've seen posted here.

Except, and this is really the fundamental catch, it's not so much "enhance" as it is "project a believable substitute/interpretation". You fundamentally can't get back information that has been destroyed/or never captured in the first place. What you can do is fill in the gaps/information with plausible values. I don't know whether this sounds like I'm splitting hairs, but it's really important that the general publ…

>we're interpolating or projecting information that is not there

But that's not fully accurate either. Sometimes the information in total will really be a more accurate representation of reality than the blurred image. Maybe it could be described as an educated guess, sometimes wrong, sometimes invaluable.

It would be interesting to see the results starting with higher quality images. With the camera quality increasing, many times there should be more data to start with.

A

Re: PixelNN – Example-Based Image Synthesis

#96

Post by WhitneyLand »

omtinez wrote: post #19
ZeroGravitas wrote: post #8

Earlier quoted context omitted.

To give specific examples from their test data, it added stubble to people who didn't have stubble, gave them a different shape of glasses, changed the color of cats, changed the color and brand of sport shoe. And even then, I'm a little suspicious of how close some of the images got to original without being given color information. It appears that info was either hidden in the original in a way not apparent to huma…

I haven't read the paper in full detail, but reading between the lines I'm guessing that there's a significant portion of manual processing and hand waving involved. From the abstract, emphasis mine: > the second stage uses a pixel-wise nearest neighbor method to map the smoothed output to multiple high-quality, high-frequency outputs in a controllable manner. My interpretation is that they select training data by ha…

There's nothing weak or negative about that, it's exactly what'd you expect. Obviously for a given input there will be multiple plausible outputs. With any such system it would make sense to allow some control in choosing among the outputs.

Re: PixelNN – Example-Based Image Synthesis

#97

Post by phkahler »

doomlaser wrote: post #44
SeanDav wrote: post #34

Earlier quoted context omitted.

Agree. This appears to be more a clever implementation of an algorithm generating "artistic" impressions. In some cases, creating artifacts which simply were not part of the original picture.

The term in neural net research is 'face hallucination': https://people.csail.mit.edu/celiu/FaceHallucination/fh.html Take a low resolution input image, and hallucinate a higher resolution version by statistically assembling bits from similar images in a large data set of training images.

If anyone ever tries to use this in court I hope they call it "Face Hallucination" and not "Image Reconstruction". On the research side, I wonder what the point of this is. I find it interesting but of little practical value.

Re: PixelNN – Example-Based Image Synthesis

#100

Post by smelterdemon »

thaw13579 wrote: post #5
otp124 wrote: post #3

I used to roll my eyes at crime television shows, whenever they said "Enhance" for a low quality image. Now it seems the possibility of that becoming realistic are increasing with a steady clip, based on this paper and other enhancement techniques I've seen posted here.

Approaches like these are hallucinating the high resolution images though--not something that we'd ever want being used for police work. That said, I wonder if it would perform better than eyewitness testimony...

I wouldn't want to see it used as evidence in court (and I doubt it would be allowed anyway but IANAL) but I could see this being a useful in certain circumstances for generating the photo-realistic equivalent of a police sketch e.g. if you had low-res security footage of a suspect and an eyewitness to guide the output.
Post reply on HN