Live data from Hacker News

Restoration of defocused and blurred images (2012)

yuzhikov.com

31–40 of 48 posts

Re: Restoration of defocused and blurred images (2012)

#31
post #29

Note that this works for out of focus photos, not for enlarging tiny details of in focus photos. They may look similarly blurry, but are mathematically very different.

How's that? Can't you think of a pixel as the average value of it's subpixels (=blur)?

You can! The problem in that case is it's not a convolution (blur) per se. Each pixel is an average of subpixels P(1)=S1(1)+S2(1)+...+Sn(1), P(2)=S1(2)+...+Sn(2) -- but as you can see there are no elements in common between pixels, which is the case with convolution. That is, in the deconvolution case there are as many variables as unknowns, whereas in the upscaling case you're creating unknowns out of a single variable.

To estimate those subpixels then you're going to be forced to make additional assumptions; if you assume they are independent you would simply estimate Sk(j)=P(j). The traditional (easiest) assumption is that the image is somewhat "bandlimited" -- it does not have many variations (frequencies) faster than once per pixel (so no equal or faster than subpixel variation). If this were the case, you could reconstruct the subpixels perfectly [1], save for some noise. But this is not always the case (and fails spectacularly when you have edges), resulting in upscale blur. So simple linear upscaling algorithms by reaching a compromise between blur and edge enhancing.

If you want to do better though, you have to use non-linear kernels and have good underlying models for your image content. A promising approach is to use machine learning/NNs: http://engineering.flipboard.com/2015/05/scaling-convnets/

[1] https://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampli...

Re: Restoration of defocused and blurred images (2012)

#32

How does an upscaler in your TV work? How does it create information out of nothing? Imagine that you are learning a deep neural network on a huge amount of movies. You have access to all of the lovely Hollywood movies. You downscale them to a 480p resolution, and then try to learn a deep neural network to upscale the thing, maybe upscaling only 16x16 blocks of the image. It works amazingly well, and looks like magic…

> you could definitely learn a filter that would be very precise in reconstructing the image when zoomed

Yes, your result would be a very clear image of one possible license plate. An algorithm may be able to do slightly better than a squinting human, but ultimately you can't retrieve destroyed information.

Re: Restoration of defocused and blurred images (2012)

#33
post #25
post #22

Strangely enough, the author only mentions total variation denoising in passing as a feature of SmartBlur. I would say this method is one of the most common, especially when your image has sharp transitions and lots of solid regions of color (e.g. pictures of buildings). I wrote what is effectively one of two of the fastest TV denoising algorithms and implementations out there: https://github.com/tansey/gfl The way t…

It wasn't too clear but I wonder if the author was referring to deconvolution under a Total Variation prior -- this is a little different to deconvolving and then applying TV denoising or just applying TV denoising. Either way, the results of overdoing it with TV are the same: cartoony images with large regions of constant colour. The difference is that incorporating TV within iterative deconvolution reduces some com…

I agree that the staircasing effect is definitely the biggest drawback of Total Variation. In the "Smoothed" picture the noise is removed but the results are blocky.

The first way to deal with it is to take into account higher powers of the differences, e.g. using a linear combination p-norms or a Huber function.

The second way is to take into account second order differences. This promotes piecewise affine instead of piecewise constant functions. You can go further and look at third order differences, but the improvement is minimal.

Other than being more complex, the biggest downside is that all of these methods have some new parameter(s) to tune.

Re: Restoration of defocused and blurred images (2012)

#34
post #32

How does an upscaler in your TV work? How does it create information out of nothing? Imagine that you are learning a deep neural network on a huge amount of movies. You have access to all of the lovely Hollywood movies. You downscale them to a 480p resolution, and then try to learn a deep neural network to upscale the thing, maybe upscaling only 16x16 blocks of the image. It works amazingly well, and looks like magic…

> you could definitely learn a filter that would be very precise in reconstructing the image when zoomed Yes, your result would be a very clear image of one possible license plate. An algorithm may be able to do slightly better than a squinting human, but ultimately you can't retrieve destroyed information.

Of course you can't retrieve destroyed information.

But information is not destroyed by perfectly unpredictable (uniform) distributions of noise.

> Yes, your result would be a very clear image of one possible license plate.

Fortunately there are methods of evaluating how well your statistical filter works, if it's precise enough you'd be fine with indeterministic nature of your filter. Or even better, you could generate all of the highly probable licence plates, instead of having only one - given by your deterministic algorithm.

Re: Restoration of defocused and blurred images (2012)

#35

How does an upscaler in your TV work? How does it create information out of nothing? Imagine that you are learning a deep neural network on a huge amount of movies. You have access to all of the lovely Hollywood movies. You downscale them to a 480p resolution, and then try to learn a deep neural network to upscale the thing, maybe upscaling only 16x16 blocks of the image. It works amazingly well, and looks like magic…

From what we know about the software on these TVs, upscaling there works using whatever code the cut-rate programmer could 1) google and 2) efficiently integrate into the product with minimal fuss.

Re: Restoration of defocused and blurred images (2012)

#36

How does an upscaler in your TV work? How does it create information out of nothing? Imagine that you are learning a deep neural network on a huge amount of movies. You have access to all of the lovely Hollywood movies. You downscale them to a 480p resolution, and then try to learn a deep neural network to upscale the thing, maybe upscaling only 16x16 blocks of the image. It works amazingly well, and looks like magic…

From what we know about the software on these TVs, upscaling there works using whatever code the cut-rate programmer could 1) google and 2) efficiently integrate into the product with minimal fuss.

Not really. Upscaling is hardware, not some ridiculously slow software solution.

https://community.sony.co.uk/t5/blog-news-from-sony/inside-4...

Upscaling is a very state of the art technology, not some layman's solution.

There are firms that specifically targeted upscaling as their product and made millions with their state-of-the-art tech. Currently upscaling is in the rise again with 4K TVs. Back in the days they made some incredible chip solutions, sold them expensively to Sony, Samsung and similar. Sony realized they can, with all of their resources (super-HD movie database) make incredible upscalers.

Just imagine that Sony has stored whole movie The Walk (distributed by Sony Pictures) in your TV in 4K resolution, the moment this movie is displayed on your screen through some lower resolution sources, they find it in the database and display the 4K content. Of course, that's highly inneficient and memory intensive, thus, they use statistical models to efficiently store movie material and have fast chips to quickly approximate the real upscale.

This will then, if the sample (number of movies) is high enough, work well on all of the movie content.

Re: Restoration of defocused and blurred images (2012)

#37

Are there security implications? If the last example was a blurred out license key or address for instance, this technique might be able to restore it.

Blurring or mosaicing is not very secure at all [1]. To be more safe block out sensitive information with solid black. (Just remember to not leak any information in the metadata, like an embedded thumbnail.) [1]: https://dheera.net/projects/blur

Woah, lesson learned. Thanks for the link.

Re: Restoration of defocused and blurred images (2012)

#40
post #29

Note that this works for out of focus photos, not for enlarging tiny details of in focus photos. They may look similarly blurry, but are mathematically very different.

How's that? Can't you think of a pixel as the average value of it's subpixels (=blur)?

If the information that belongs in one pixel is spread out over 50x50 pixels, according a known mathematical formula, you can reconstruct it pretty well.

If you just have one pixel with the average of what it covers, then that is all you have.

Post reply on HN