Neural Enhance – Super Resolution for images using deep learning • Hacker News Board

Re: Neural Enhance – Super Resolution for images using deep learning

#11

Post by antome » Sat, Oct 29, 2016, 8:52 AM UTC

dharma1 wrote: post #4

nullc wrote: post #2

Comparison using nearest neighbor, instead of a more reasonable linear filter, or-- heaven forbid-- some edge basic directed interpolator... is a little cheaty.

Agreed, it would have been nice to show other upscaling algorithms. But neural net super resolution generators can still have significantly more detail at 4-8x, as shown here http://arxiv.org/abs/1609.04802

Really cool to see them taking a page out of X264, trying to match the "energy" of the image rather than the "correctness".

Re: Neural Enhance – Super Resolution for images using deep learning

#12

Post by shash7 » Sat, Oct 29, 2016, 9:42 AM UTC

Granted it won't be actually sharpening the images but for 99% of the use cases it would be awesome!

Re: Neural Enhance – Super Resolution for images using deep learning

#13

Post by Leynos » Sat, Oct 29, 2016, 10:10 AM UTC

A friend of mine suggested that an approach similar to this could be used to upscale old standard definition TV shows (specifically, those shot on video rather than film). I'd imagine that multiple specially trained networks would be employed for different parts of the image (trained on pictures of individual performers or types of set/background). Pleased to see that this is possible. Is there anyone doing something along those lines already?

Re: Neural Enhance – Super Resolution for images using deep learning

#14

Post by thenomad » Sat, Oct 29, 2016, 10:14 AM UTC

This looks amazing.

Question for the more experienced deep learning folk: if I wanted to use this to upscale textures for a game, would I have to train it on the same type of texture? In other words additional wood textures when upscaling wood, brick when upscaling brick textures, and so on?

Re: Neural Enhance – Super Resolution for images using deep learning

#15

Post by placebo » Sat, Oct 29, 2016, 10:22 AM UTC

It definitely makes a significant qualitative improvement, making the picture appear more in sync with what our brain interprets as a higher resolution picture, but my first thought is whether this particular example goes beyond aesthetics. Is there really any instance where this method could for instance turn an unintelligible picture of a license plate to something in which the characters can be recognised? More generally, I wonder whether there has been any research on the limits - i.e, what needs to be the combined minimal size of the information stored in the neural network plus the information on its inputs before the output can be said to be true to the source with probability x ?

Re: Neural Enhance – Super Resolution for images using deep learning

#16

Post by martinkallstrom » Sat, Oct 29, 2016, 10:47 AM UTC

thenomad wrote: post #14

This looks amazing. Question for the more experienced deep learning folk: if I wanted to use this to upscale textures for a game, would I have to train it on the same type of texture? In other words additional wood textures when upscaling wood, brick when upscaling brick textures, and so on?

Yes, that would help a lot with output quality. The machine can only hallucinate what it has previously seen.

Re: Neural Enhance – Super Resolution for images using deep learning

#17

Post by Mithaldu » Sat, Oct 29, 2016, 10:56 AM UTC

Leynos wrote: post #13

A friend of mine suggested that an approach similar to this could be used to upscale old standard definition TV shows (specifically, those shot on video rather than film). I'd imagine that multiple specially trained networks would be employed for different parts of the image (trained on pictures of individual performers or types of set/background). Pleased to see that this is possible. Is there anyone doing something…

It should also be possible to train it on itself to improve moving scenes by using the motion itself as temporal super-sampling, just like the human eye does.

Re: Neural Enhance – Super Resolution for images using deep learning

#18

Post by dr_zoidberg » Sat, Oct 29, 2016, 11:21 AM UTC

ENGNR wrote: post #10

We enhanced the image like on CSI and look, the defendants face! "Because my photos were used heavily in the dataset..." Jury: So guilty

Defense should train a network with faces of the jury and then show how the same technique, run by their biased network, now shows each of them in the scene of the crime :)

Re: Neural Enhance – Super Resolution for images using deep learning

#19

Post by ttul » Sat, Oct 29, 2016, 11:30 AM UTC

Mithaldu wrote: post #17

Leynos wrote: post #13

A friend of mine suggested that an approach similar to this could be used to upscale old standard definition TV shows (specifically, those shot on video rather than film). I'd imagine that multiple specially trained networks would be employed for different parts of the image (trained on pictures of individual performers or types of set/background). Pleased to see that this is possible. Is there anyone doing something…

It should also be possible to train it on itself to improve moving scenes by using the motion itself as temporal super-sampling, just like the human eye does.