All those examples are fairly low-resolution. Does this approach scale or can it be applied in some tiled fashion? Or would the artifacts get worse for larger images?
PixelNN – Example-Based Image Synthesis
41–50 of 155 posts
Re: PixelNN – Example-Based Image Synthesis
#42I used to roll my eyes at crime television shows, whenever they said "Enhance" for a low quality image. Now it seems the possibility of that becoming realistic are increasing with a steady clip, based on this paper and other enhancement techniques I've seen posted here.
Except, and this is really the fundamental catch, it's not so much "enhance" as it is "project a believable substitute/interpretation". You fundamentally can't get back information that has been destroyed/or never captured in the first place. What you can do is fill in the gaps/information with plausible values. I don't know whether this sounds like I'm splitting hairs, but it's really important that the general publ…
> Except, and this is really the fundamental catch, it's not so much "enhance" as it is "project a believable substitute/interpretation".
I would argue that this is a form of enhancement though, and in some cases will be enough to completely reconstruct the original image. For example, if I give you a scanned PDF, and you know for a fact that it was size 12 black Ariel text on a white background, this can feasibly let you reconstruct the original image perfectly. The 'prior' that has been encoded by the model from the large amount of other images increases the mutual information between grainy image and high-res. The catch is that uncertainty cannot be removed entirely, and you need to know that the target image comes from roughly the same distribution as the training set. But knowing this gives you information that is not encoded in the pixels themselves, so you can't necessarily argue that some enhancement is impossible. For example with celebrity images, if the model is able to figure out who is in the picture, this massively decreases the set of plausible outputs.
Re: PixelNN – Example-Based Image Synthesis
#43I used to roll my eyes at crime television shows, whenever they said "Enhance" for a low quality image. Now it seems the possibility of that becoming realistic are increasing with a steady clip, based on this paper and other enhancement techniques I've seen posted here.
Except, and this is really the fundamental catch, it's not so much "enhance" as it is "project a believable substitute/interpretation". You fundamentally can't get back information that has been destroyed/or never captured in the first place. What you can do is fill in the gaps/information with plausible values. I don't know whether this sounds like I'm splitting hairs, but it's really important that the general publ…
The benefit depends on how predictable the phenomenon is that your are interpolating from. Sometimes it will be quantitatively better than a low resolution version, sometimes not.
A good example is with compression algorithms for media. They work because the sound or image is predictable. And they are ineffective when the input becomes more unpredictable. But if the output is all you have then running the decompression will probably be better than just reading the raw compressed data. But you have to be aware of the limitations.
Re: PixelNN – Example-Based Image Synthesis
#44I found the title somewhat misleading. I was expecting some clever application of the nearest-neighbor interpolation. But this seems to involve neural nets and appears far from "simple" to me (I'm not in the image processing field though).
Agree. This appears to be more a clever implementation of an algorithm generating "artistic" impressions. In some cases, creating artifacts which simply were not part of the original picture.
The term in neural net research is 'face hallucination': https://people.csail.mit.edu/celiu/FaceHallucination/fh.html
Take a low resolution input image, and hallucinate a higher resolution version by statistically assembling bits from similar images in a large data set of training images.
Re: PixelNN – Example-Based Image Synthesis
#45I used to roll my eyes at crime television shows, whenever they said "Enhance" for a low quality image. Now it seems the possibility of that becoming realistic are increasing with a steady clip, based on this paper and other enhancement techniques I've seen posted here.
Except, and this is really the fundamental catch, it's not so much "enhance" as it is "project a believable substitute/interpretation". You fundamentally can't get back information that has been destroyed/or never captured in the first place. What you can do is fill in the gaps/information with plausible values. I don't know whether this sounds like I'm splitting hairs, but it's really important that the general publ…
The white shoe output vs black shoe output illustrates this fairly well.
Re: PixelNN – Example-Based Image Synthesis
#46So is there an analagous process that would apply to audio I wonder?
What would the lo-res starting point be? Low sample-rate, bit depth, ...?
Look up compressed sensing for audio.
(Eg first result: http://sunbeam.ece.wisc.edu/csaudio/)
Re: PixelNN – Example-Based Image Synthesis
#47Earlier quoted context omitted.
It can give possible matches, i don't think it would be admissible in court. they could still trick a confession out of someone using that image.
It could also narrow down the list of suspects. From there, additional investigation can find more evidence. Having access to big data can help this.
true, it cannot be used to "nail" a perp tho, just to help gain extra evidence.
Re: PixelNN – Example-Based Image Synthesis
#48I hope some day this will generalize to video. I don't care about the exact shape of background trees in an action movie - with this approach, they could be compressed to just a few bytes, regardless of resolution.
That's what video compression does now.
No, today's compression is about compressing what's already in the one movie. But imagine that you run your training set over 100's or 1000's of films, and extract just enough to represent say different types of trees in a few bytes. You could 'compress' a film by replacing data with markers that essentially describe some properties of the tree, and those properties + the training set are then used during 'decompression' to recreate (an approximation of) the tree.
This would of course not give you any space savings when you want to distribute 1 movie. There would be some minimum number of movies where the training set + actual movies would be smaller than the sum of the sizes of the individual movies compressed.
I'm not saying this would be a net space saver, or necessarily a good technique at all, but the concept is intriguing.
Re: PixelNN – Example-Based Image Synthesis
#49No code available.
Re: PixelNN – Example-Based Image Synthesis
#50Earlier quoted context omitted.
It could also narrow down the list of suspects. From there, additional investigation can find more evidence. Having access to big data can help this.
true, it cannot be used to "nail" a perp tho, just to help gain extra evidence.
Yup. In a court of law, the value as evidence is going to be weighted fairly low, even with expert testimony. It may be enough to get a warrant, or a piece in the process of deduction during the investigation phase.