Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PixelNN – Example-Based Image Synthesis

cs.cmu.edu

Post reply on HN
Print view View original

121–130 of 155 posts

Re: PixelNN – Example-Based Image Synthesis

#121

Post by ComputerGuru »

chrismorgan wrote: post #68
ComputerGuru wrote: post #12

Earlier quoted context omitted.

> you only have one value that encodes to a certain hash value, right? Errr wrong. A perfect hash, yes. But they're never perfect. You have a collision domain and you hope that you don't have enough inputs to trigger a birthday paradox. Look at the pictures on the article. It's an outline of the shoe. That's your hash. ANY shoe with that general outline resolves to that same hash. If your input is objects found in th…

FYI (not because it’s particularly relevant to the sort of hashing that is being talked about, but because it’s a useful piece of info that might interest people, and corrects what I think is a misunderstanding in the parent comment): perfect hash functions are a thing, and are useful: https://en.wikipedia.org/wiki/Perfect_hash_function . So long as you’re dealing with a known, finite set of values, you can craft a u…

Yeah, I debated bringing it up but since we were in the context of not knowing set members ahead of time, decided not to.

Thanks for the rust-phf link. I'm bookmarking for my next project!

Re: PixelNN – Example-Based Image Synthesis

#122

Post by mlwelles »

I noticed that all of the human examples are caucasian. I'd be very interested to see how accurate it is with more representative range of human faces than how it handles animals or handbags.

Had personal experience on a project where the facial scanning engine failed spectacularly when anyone except white men like me tried to use it.

An experience that's pretty common, too:

https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&c...

http://www.telegraph.co.uk/technology/2017/08/09/faceapp-spa...

https://www.theatlantic.com/technology/archive/2016/04/the-u...

https://www.theguardian.com/technology/2017/may/28/joy-buola...

Re: PixelNN – Example-Based Image Synthesis

#123

Post by sctb »

avian wrote: post #25

I found the title somewhat misleading. I was expecting some clever application of the nearest-neighbor interpolation. But this seems to involve neural nets and appears far from "simple" to me (I'm not in the image processing field though).

We've updated the title from the submitted “Simple Nearest-Neighbor Approach Creates Photorealistic Image from Low-Res Image” to that of the article.

Re: PixelNN – Example-Based Image Synthesis

#124

Post by dahart »

jopsen wrote: post #6
ACow_Adonis wrote: post #4

Earlier quoted context omitted.

Except, and this is really the fundamental catch, it's not so much "enhance" as it is "project a believable substitute/interpretation". You fundamentally can't get back information that has been destroyed/or never captured in the first place. What you can do is fill in the gaps/information with plausible values. I don't know whether this sounds like I'm splitting hairs, but it's really important that the general publ…

Sometimes US justice system seems very "approximate". So why not convict people based on interpolated evidence? - I'm joking of course :) hehe

Sadly, it actually happens sometimes.

https://www.wired.com/2017/04/courts-using-ai-sentence-crimi...

This thread a year ago worried about it too, but the paper itself seems implausible and problematic.

https://news.ycombinator.com/item?id=12983827

Re: PixelNN – Example-Based Image Synthesis

#125

Post by dahart »

scarface74 wrote: post #29
ACow_Adonis wrote: post #4

Earlier quoted context omitted.

Except, and this is really the fundamental catch, it's not so much "enhance" as it is "project a believable substitute/interpretation". You fundamentally can't get back information that has been destroyed/or never captured in the first place. What you can do is fill in the gaps/information with plausible values. I don't know whether this sounds like I'm splitting hairs, but it's really important that the general publ…

What you can do though, in limited circumstances, is create a still picture with more detail from a lower quality video. https://photo.stackexchange.com/questions/17098/csi-image-re...

I seem to remember from my computer vision class way back when that there's a fundamental theoretical limit to the amount of detail you can get out of a moving sequence. Recovering frequencies a little higher than the pixel sampling is definitely possible, but I feel like it was maybe something like 10x theoretical maximum. I also get the feeling, from looking around at available software, that in practice achieving 2-3x is the most you can get in ideal conditions, and most video is far from ideal.

Re: PixelNN – Example-Based Image Synthesis

#126

Post by jaclaz »

ComputerGuru wrote: post #12
gambiting wrote: post #10

Earlier quoted context omitted.

No, but think of these blurred images as a "hash" - in an ideal situation, you only have one value that encodes to a certain hash value, right? So If you are given a hash X you technically can work out that it was derived from value Y - you're not getting back information that was lost - in a way it was merely encoded into the blurred image, and it should be possible to produce a real image which, when blurred, will…

> you only have one value that encodes to a certain hash value, right? Errr wrong. A perfect hash, yes. But they're never perfect. You have a collision domain and you hope that you don't have enough inputs to trigger a birthday paradox. Look at the pictures on the article. It's an outline of the shoe. That's your hash. ANY shoe with that general outline resolves to that same hash. If your input is objects found in th…

>So back to the blurry security camera footage of a license plate or a face. Sure, that "hash" can reliably tell you that it wasn't a sasquatch that committed the robbery, but it literally doesn't contain the data necessary to _ever_ prove it was the suspect in question, even if the techs _can_ prove that the suspect hashes to the image in the footage.

For a face, sure, for printed text/license plates there are effective deblurring algorithms that in some cases may rebuild a readable image.

A (IMHO good) software is this one (was freeware, now it is Commercial, this is the last freeware version):

https://github.com/Y-Vladimir/SmartDeblur/downloads

You can try it (just for the fun of it) on these two images:

https://articles.forensicfocus.com/2014/10/08/can-you-get-th...

https://forensicfocus.files.wordpress.com/2014/09/out-of-foc...

https://forensicfocus.files.wordpress.com/2014/09/moving-car...

For the first choose "Out of Focus Blur" and play with the values, you should get a decent image at roughly Radius 8, Smooth 40%, Correction Strength 0%, Edge Feather 10%

For the second choose "motion Blur" and play with the values, you should get a decent image at roughly Length 14, Angle 34, Smooth 50%,

Re: PixelNN – Example-Based Image Synthesis

#127

Post by jampekka »

dahart wrote: post #103
avian wrote: post #25

I found the title somewhat misleading. I was expecting some clever application of the nearest-neighbor interpolation. But this seems to involve neural nets and appears far from "simple" to me (I'm not in the image processing field though).

> I was expecting some clever application of the nearest-neighbor interpolation. But this seems to involve neural nets and appears far from "simple" to me (I'm not in the image processing field though). It's not that far off actually, but they are talking about nearest neighbor Markov chains, not interpolation. You probably already know nearest neighbor Markov chains because there are lots of text examples, and a ton…

I don't think this method has anything to do with Markov Chains. The spatial structure isn't explicitly used at all, and the interpolation/regression is quite a vanilla nearest neighbor with some performance tricks.

Well, of course almost anything can be interpreted as a Markov process, but I don't think it's a very useful abstraction here.

Re: PixelNN – Example-Based Image Synthesis

#128

Post by kevin_thibedeau »

ACow_Adonis wrote: post #4
otp124 wrote: post #3

I used to roll my eyes at crime television shows, whenever they said "Enhance" for a low quality image. Now it seems the possibility of that becoming realistic are increasing with a steady clip, based on this paper and other enhancement techniques I've seen posted here.

Except, and this is really the fundamental catch, it's not so much "enhance" as it is "project a believable substitute/interpretation". You fundamentally can't get back information that has been destroyed/or never captured in the first place. What you can do is fill in the gaps/information with plausible values. I don't know whether this sounds like I'm splitting hairs, but it's really important that the general publ…

"Ladies and gentlemen of the jury, we will definitively prove that the black smudge captured on camera was in fact a gun"

Already being done today with DNA.

Re: PixelNN – Example-Based Image Synthesis

#129

Post by pedrosorio »

amelius wrote: post #79
matt4077 wrote: post #55

Earlier quoted context omitted.

> You fundamentally can't get back information that has been destroyed/or never captured in the first place. I love this cliché. I've seen it thousands of times, and probably written it myself a few times. We all repeat stuff like that ad nauseam, without ever thinking. Because it's fundamentally flawed, especially in the context that it has usually been applied to, namely criticising the CSI:XYZ trope of "enhancing…

> It's similar to that guy who used Photoshop's swirl effect to hide his face, not knowing that the effect is deterministic, and can easily be undone. The effect does not only need to be deterministic, but also invertible. A low-res image has multiple "inverses" (yikes), supposedly each with an associated probability (if you would model it that way). So it would be more honest if the algorithm shows them all.

It's what they call an https://en.wikipedia.org/wiki/Inverse_problem

Re: PixelNN – Example-Based Image Synthesis

#130

Post by dahart »

jampekka wrote: post #127
dahart wrote: post #103

Earlier quoted context omitted.

> I was expecting some clever application of the nearest-neighbor interpolation. But this seems to involve neural nets and appears far from "simple" to me (I'm not in the image processing field though). It's not that far off actually, but they are talking about nearest neighbor Markov chains, not interpolation. You probably already know nearest neighbor Markov chains because there are lots of text examples, and a ton…

I don't think this method has anything to do with Markov Chains. The spatial structure isn't explicitly used at all, and the interpolation/regression is quite a vanilla nearest neighbor with some performance tricks. Well, of course almost anything can be interpreted as a Markov process, but I don't think it's a very useful abstraction here.

> I don't think this method has anything to do with Markov Chains.

Oh, it absolutely does. I think it's fair to say that Efros launched the field of nearest neighbor texture synthesis, and his abstract states: "The texture synthesis process grows a new image outward from an initial seed, one pixel at a time. A Markov random field model is assumed, and the conditional distribution of a pixel given all its neighbors synthesized so far is estimated by querying the sample image and finding all similar neighborhoods.

This is the same Markov model that all subsequent texture synthesis papers are implicitly using, including the paper at the top of this thread. Efros' paper implemented directly is really slow, so a huge number of subsequent papers use the same conceptual framework, and are only adding methods for making the method performant and practical. (Sometimes, at the cost of some quality -- many cannot synthesize text, for example.)

Note the inspiration for text synthesis, Shannon's paper, also describes the "Markoff Process" explicitly. http://math.harvard.edu/~ctm/home/text/others/shannon/entrop... (Efros referenced Shannon, and noted on his web page: "Special thanks goes to Prof. Joe Zachary who taught my undergrad data structures course and had us implement Shannon's text synthesis program which was the inspiration for this project.")

> Well, of course almost anything can be interpreted as a Markov process, I don't think it's a very useful abstraction here.

It's not an abstraction to build a conditional probability table and then sample from it repeatedly to synthesize a new output. That's what a Markov process is, and that's what the paper posted here is doing. I don't really understand why you feel it's distant and abstract, but if you want to elaborate, I am willing to listen!

Post reply on HN