Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Neural Enhance – Super Resolution for images using deep learning

github.com

Post reply on HN
Print view View original

41–50 of 53 posts

Re: Neural Enhance – Super Resolution for images using deep learning

#41

Post by nuclai »

WhitneyLand wrote: post #27

This approach can be equaled or bettered with no machine learning. This example allows easy comparison between common techniques. Choose image 7 to see an example with a person: https://dl.dropboxusercontent.com/u/2810224/Homepage/publica...

(Author here.) Did you see the faces example on the GitHub page? It was a domain-specific network trained adversarially for that purpose, but I have yet to see any super-resolution of that quality with or without machine learning.

Most other approaches don't even try to inject high-frequency detail into the high-resolution images because the PSNR/SSIM benchmarks drop. Until those metrics/benchmarks are dropped, there'll be little more progress in super-resolution.

Re: Neural Enhance – Super Resolution for images using deep learning

#42

Post by nuclai »

thenomad wrote: post #14

This looks amazing. Question for the more experienced deep learning folk: if I wanted to use this to upscale textures for a game, would I have to train it on the same type of texture? In other words additional wood textures when upscaling wood, brick when upscaling brick textures, and so on?

(Author here.) If you have the luxury to train on domain-specific textures, the results will definitely be better. That's why I included all the training code in the repository as well—to allow for this kind of solution.

If you scroll down on GitHub to see the faces examples, those are achieved by a domain-specific network. I suspect you'll similarly get extremely high-quality if you have good input images.

Re: Neural Enhance – Super Resolution for images using deep learning

#43

Post by nuclai »

shash7 wrote: post #12

Granted it won't be actually sharpening the images but for 99% of the use cases it would be awesome!

(Author here.) Unlike most other non generative adversarial network (GAN) approaches to super-resolution, it does try to inject high-frequency detail; see the faces example on GitHub. But I tuned down that parameter in the released models a bit so it performed better generally.

Re: Neural Enhance – Super Resolution for images using deep learning

#44

Post by nuclai »

kuschku wrote: post #25
andai wrote: post #23

Earlier quoted context omitted.

Wow how much do you have?

On this system I’ve got 32GB, of which about 2GB were used by the OS itself, and another 2GB by firefox, that’s why it stopped at around 28GB.

(Author here.) Maybe it's worth moving to a GitHub issue. Try `--model=small`. The demo server limits the number of pixels to around 320x200 or 256x256 and can do only 4 at the same time to fit in RAM.

Re: Neural Enhance – Super Resolution for images using deep learning

#45

Post by placebo »

PeterisP wrote: post #31
placebo wrote: post #15

It definitely makes a significant qualitative improvement, making the picture appear more in sync with what our brain interprets as a higher resolution picture, but my first thought is whether this particular example goes beyond aesthetics. Is there really any instance where this method could for instance turn an unintelligible picture of a license plate to something in which the characters can be recognised? More ge…

Well, it doesn't create any information that wasn't in the original data (nothing can do that, you can only lose information in processing) so if e.g. the characters can be recognized in the processed image of a licence plate, then by definition they could have been recognized from the original data as well in some manner. However, they can make things more easily interpretable by humans . A rough analogy is turning…

> Well, it doesn't create any information that wasn't in the original data (nothing can do that, you can only lose information in processing)

I'm not sure this is correct. In a sense, it does contain information that wasn't in the original inputs - i.e information added by the weights in the neural network which itself was obtained by information extracted from an enormous amount of previous samples. Of course, the largest and best trained neural network won't be able to tell the license number given 2 pixels of information, but I am curious as to the theoretical limits of what can be achieved in extreme cases of with very little information as input and a neural network that has almost limitless resources.

Re: Neural Enhance – Super Resolution for images using deep learning

#47

Post by webmaven »

nuclai wrote: post #37
manav wrote: post #36

I've seen a number of neural network approaches for super-resolution like waifu, but I haven't seen something general purpose thats better than bicubic/fourier/nearest neighbor. Would be nice if the author did a comparison.

(Author here.) My biggest insight from this project is that super-resolution with neural networks benefits significantly from being domain specific. If you train on broader datasets, it does pretty well but has to make compromises. Many recent papers do a comparison in terms of pixel similarity (PSNR/SSIM), and using those metrics the quality drops because high-frequency detail is punished under those criteria (even…

> super-resolution with neural networks benefits significantly from being domain specific. If you train on broader datasets, it does pretty well but has to make compromises.

To what extent could the need for this trade-off be overcome with a larger network?

Re: Neural Enhance – Super Resolution for images using deep learning

#48

Post by nuclai »

beautifulfreak wrote: post #22

I do photo restorations on Reddit, where people often submit blurry photos that sharpening just can't fix. It would be great if this were offered as an online service.

Yes, it sounds possible with this code — but would require training a new network. Do you have a link to some examples?

Re: Neural Enhance – Super Resolution for images using deep learning

#49

Post by mm_alex »

Mithaldu wrote: post #17
Leynos wrote: post #13

A friend of mine suggested that an approach similar to this could be used to upscale old standard definition TV shows (specifically, those shot on video rather than film). I'd imagine that multiple specially trained networks would be employed for different parts of the image (trained on pictures of individual performers or types of set/background). Pleased to see that this is possible. Is there anyone doing something…

It should also be possible to train it on itself to improve moving scenes by using the motion itself as temporal super-sampling, just like the human eye does.

this works quite well, and does not necessarily require any NN/machine learning. see the youtube for this paper https://www.disneyresearch.com/publication/scenespace/ tldr simple brute force weighted average of samples from many frames, combined with a noisy/low quality depth-from-motion estimate can be used to de-noise, increase resolution and otherwise manipulate video footage. very cool paper with great results from a simple technique.

Re: Neural Enhance – Super Resolution for images using deep learning

#50

Post by beautifulfreak »

nuclai wrote: post #48
beautifulfreak wrote: post #22

I do photo restorations on Reddit, where people often submit blurry photos that sharpening just can't fix. It would be great if this were offered as an online service.

Yes, it sounds possible with this code — but would require training a new network. Do you have a link to some examples?

Deblurring requests turn up frequently on https://www.reddit.com/r/estoration/ and https://www.reddit.com/r/picrequests/
Post reply on HN