Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Neural Enhance – Super Resolution for images using deep learning

github.com

Post reply on HN
Print view View original

31–40 of 53 posts

Re: Neural Enhance – Super Resolution for images using deep learning

#31

Post by PeterisP »

placebo wrote: post #15

It definitely makes a significant qualitative improvement, making the picture appear more in sync with what our brain interprets as a higher resolution picture, but my first thought is whether this particular example goes beyond aesthetics. Is there really any instance where this method could for instance turn an unintelligible picture of a license plate to something in which the characters can be recognised? More ge…

Well, it doesn't create any information that wasn't in the original data (nothing can do that, you can only lose information in processing) so if e.g. the characters can be recognized in the processed image of a licence plate, then by definition they could have been recognized from the original data as well in some manner.

However, they can make things more easily interpretable by humans. A rough analogy is turning up the contrast - given a very dark image of licence plate where the black parts are totally black (#000000) and white parts are just very dark (#010101), the characters definitely can be recognized even while human in normal conditions would just see it as totally black, and processing would help.

Re: Neural Enhance – Super Resolution for images using deep learning

#32

Post by osense »

andai wrote: post #21
ingenter wrote: post #20

Here's a list of various image interpolation techniques, with similar goal: - http://www.wisdom.weizmann.ac.il/~vision/SingleImageSR.html - http://chiranjivi.tripod.com/EDITut.html - http://www.tecnick.com/pagefiles/appunti/iNEDI_tesi_Nicola_A... - http://www.eurasip.org/Proceedings/Eusipco/Eusipco2009/conte... - http://bengal.missouri.edu/~kes25c/ http://bengal.missouri.edu/~kes25c/nnedi3.zip http://forum.doom9.org/…

My question is offtopic, but how do you keep lists of urls like that? Do you just use text files? I'm struggling with too much to read

I like pinboard.in

Re: Neural Enhance – Super Resolution for images using deep learning

#33

Post by oh_sigh »

andai wrote: post #21
ingenter wrote: post #20

Here's a list of various image interpolation techniques, with similar goal: - http://www.wisdom.weizmann.ac.il/~vision/SingleImageSR.html - http://chiranjivi.tripod.com/EDITut.html - http://www.tecnick.com/pagefiles/appunti/iNEDI_tesi_Nicola_A... - http://www.eurasip.org/Proceedings/Eusipco/Eusipco2009/conte... - http://bengal.missouri.edu/~kes25c/ http://bengal.missouri.edu/~kes25c/nnedi3.zip http://forum.doom9.org/…

My question is offtopic, but how do you keep lists of urls like that? Do you just use text files? I'm struggling with too much to read

The key when you have too much to read is losing links, not retaining them better.

Re: Neural Enhance – Super Resolution for images using deep learning

#34

Post by kelvin0 »

ENGNR wrote: post #10

We enhanced the image like on CSI and look, the defendants face! "Because my photos were used heavily in the dataset..." Jury: So guilty

Wow, we'll finally blow the lid off all these conspiracy theories when we 'unblur' the pictures of : Sasquatch, UFOs, Loch Ness Monster ... :)

Re: Neural Enhance – Super Resolution for images using deep learning

#35

Post by louprado »

Leynos wrote: post #13

A friend of mine suggested that an approach similar to this could be used to upscale old standard definition TV shows (specifically, those shot on video rather than film). I'd imagine that multiple specially trained networks would be employed for different parts of the image (trained on pictures of individual performers or types of set/background). Pleased to see that this is possible. Is there anyone doing something…

As you suggested, continuity of appearance is what makes this problem so difficult.

I recall watching a movie that was converted from black-and-white to color as a child. There were many distracting artifacts. Most notable was the hairlines of the actors would shift as the actor rotated their head. It made the film unwatchable.

Re: Neural Enhance – Super Resolution for images using deep learning

#37

Post by nuclai »

manav wrote: post #36

I've seen a number of neural network approaches for super-resolution like waifu, but I haven't seen something general purpose thats better than bicubic/fourier/nearest neighbor. Would be nice if the author did a comparison.

(Author here.) My biggest insight from this project is that super-resolution with neural networks benefits significantly from being domain specific. If you train on broader datasets, it does pretty well but has to make compromises. Many recent papers do a comparison in terms of pixel similarity (PSNR/SSIM), and using those metrics the quality drops because high-frequency detail is punished under those criteria (even though it may look better perceptually). Reference: http://arxiv.org/abs/1609.04802

On GitHub, below each GIF there's a demo comparison, but on the site you can also submit your own to try it out (click on title or restart button). Takes about 60s currently; running on CPU as GPUs are busy training ;-)

Re: Neural Enhance – Super Resolution for images using deep learning

#38

Post by nuclai »

louprado wrote: post #35
Leynos wrote: post #13

A friend of mine suggested that an approach similar to this could be used to upscale old standard definition TV shows (specifically, those shot on video rather than film). I'd imagine that multiple specially trained networks would be employed for different parts of the image (trained on pictures of individual performers or types of set/background). Pleased to see that this is possible. Is there anyone doing something…

As you suggested, continuity of appearance is what makes this problem so difficult. I recall watching a movie that was converted from black-and-white to color as a child. There were many distracting artifacts. Most notable was the hairlines of the actors would shift as the actor rotated their head. It made the film unwatchable.

(Author here.) Absolutely! Using multiple super-resolution networks, not only continuity would present problems, but also blending between different regions. I agree there's a lot of value for domain-specific networks here, as you can see from the faces example on GitHub.

I'd be curious to see an ensemble-based super-resolution, where each model can output the confidence of a pixel region, then have another network learn to blend the result.

Conversely, these results are achieved using a single top-of-range GPU. Everything fits in memory for a batch-size 15 at 192x192. By distributing the training somehow, you could make the network 10x bigger and train for a whole week and likely get much better general purpose results.

Re: Neural Enhance – Super Resolution for images using deep learning

#40

Post by nuclai »

dharma1 wrote: post #4
nullc wrote: post #2

Comparison using nearest neighbor, instead of a more reasonable linear filter, or-- heaven forbid-- some edge basic directed interpolator... is a little cheaty.

Agreed, it would have been nice to show other upscaling algorithms. But neural net super resolution generators can still have significantly more detail at 4-8x, as shown here http://arxiv.org/abs/1609.04802

(Author here.) Yeah, I knew this would come up but decided to proceed with the pixelated comparison anyway. I couldn't get the GIFs to reflect the results because of 8-bit quantization/dithering. The images show the neural network inputs and outputs, not a comparison with other super-resolution algorithms (still fascinating :-).

I'm working on the Docker instance now, that should help anyone with interest/experience in the field compare results easily.

Post reply on HN