I did my thesis on using medioni's tensor voting framework to reconstruct noisy, blurry, low-res and the like images. It was sponsored by USGS on a data set that I thought was a bit of a bizarre use case. The approach worked pretty well, with some reasonable success at doing "COMPUTER ENHANCE" type computer vision magic. Later on talking with my advisor about the bizarrely mundane and uninteresting data sets we were working on from the grant he quipped that "You built a reasonable way of unblurring and enhancing unreadable images, the military doesn't care about this mundane use case." It then occurred that i'd been wildly ignorant to what I just spent 2 years of my life on.
It's all a blur
51–60 of 70 posts
Re: It's all a blur
#52Blur is perhaps surprisingly one of the degradations we know best how to undo. It's been studied extensively because there's just so many applications, for microscopes, telescopes, digital cameras. The usual tricks revolve around inverting blur kernels, and making educated guesses about what the blur kernel and underlying image might look like. My advisors and I were even able to train deep neural networks using only…
Isn't that roughly (ok, very roughly) how generative diffusion AIs work when you ask them to make an image?
I believe diffusion image models learn to model a reverse-noising function, rather than reverse-blurring.
Re: It's all a blur
#53One salient point not touched on here, is that an awful lot of the time, the things folks are blurring out specifically is text. And since we know an awful lot about what text ought to look like, we have a lot more information to guide the reconstruction...
And older people are very good at reading blurry text.
(My grandmother always told me to "never get old." I wish I followed her advice.)
Re: It's all a blur
#54> This nets us another original pixel value, img(8).
This makes it all seem really too pat. In fact, this probably doesn't get us the original pixel value, because of quantizing deleting information when the blur was applied, which can never be recovered afterwards. We can at best get an approximation of the original value, which is rather obvious given that we can vaguely make out figures in a blurred image already.
> Nevertheless, even with a large averaging window, fine detail — including individual strands of hair — could be recovered and is easy to discern.
The reason for this is that he's demonstrating a box blur. A box blur is roughly equivalent to taking the frequency transform of the image, then multiplying it by a sort of decaying sin wave. This achieves a "blur" in that the lowest frequency is multiplied by 1 and hence is retained, and higher frequencies are attenuated. However, visually we can see that a box blur doesn't look very good, and importantly it doesn't necessarily attenuate the very highest frequencies by much more than far lower frequencies. Hence it isn't surprising that the highest frequencies can be recovered in good fidelity. Compare a gaussian blur, which is usually considered to look better, and whose frequency transform focuses all the attenuation at the highest frequencies. You would be far less able to recover individual strands of hair in an image that was gaussian blurred.
> Remarkably, the information “hidden” in the blurred images survives being saved in a lossy image format.
Remarkable, maybe, but unsurprising if you understand that jpeg operates on basically the same frequency logic as described above. Specifically, it will be further attenuating and quantizing the highest frequencies of the image. Since the box blur has barely attenuated them already, this doesn't affect our ability to recover the image.
Re: It's all a blur
#55> This nets us another original pixel value, img(8). This makes it all seem really too pat. In fact, this probably doesn't get us the original pixel value, because of quantizing deleting information when the blur was applied, which can never be recovered afterwards. We can at best get an approximation of the original value, which is rather obvious given that we can vaguely make out figures in a blurred image already.…
> You would be far less able to recover individual strands of hair in an image that was gaussian blurred.
Frequency-domain deconvolution is frequency-domain deconvolution, right? It doesn’t really matter what your kernel is.
Re: It's all a blur
#56Blur is perhaps surprisingly one of the degradations we know best how to undo. It's been studied extensively because there's just so many applications, for microscopes, telescopes, digital cameras. The usual tricks revolve around inverting blur kernels, and making educated guesses about what the blur kernel and underlying image might look like. My advisors and I were even able to train deep neural networks using only…
Isn't that roughly (ok, very roughly) how generative diffusion AIs work when you ask them to make an image?
You're absolutely right! Diffusion models basically invert noise (random Gaussian samples that you add independently to every pixel) but they can also work with blur instead of noise.
Generally when you're dealing with a blurry image you're gonna be able to reduce the strength of the blur up to a point but there's always some amount of information that's impossible to recover. At this point you have two choices, either you leave it a bit blurry and call it a day or you can introduce (hallucinate) information that's not there in the image. Diffusion models generate images by hallucinating information at every stage to have crisp images at the end but in many deblurring applications you prefer to stay faithful to what's actually there and you leave the tiny amount of blur left at the end.
Re: It's all a blur
#57Earlier quoted context omitted.
Isn't that roughly (ok, very roughly) how generative diffusion AIs work when you ask them to make an image?
I believe diffusion image models learn to model a reverse-noising function, rather than reverse-blurring.
Most of them do but it's not mandatory and deblurring can be used [1]
[1] Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise, Bansal et al., NeurIPS 2023
Re: It's all a blur
#58Blur is perhaps surprisingly one of the degradations we know best how to undo. It's been studied extensively because there's just so many applications, for microscopes, telescopes, digital cameras. The usual tricks revolve around inverting blur kernels, and making educated guesses about what the blur kernel and underlying image might look like. My advisors and I were even able to train deep neural networks using only…
Just to add to this: intentional/digital blur is even easier to undo as the source image is still mostly there. You just have to find the inverse metric.
This is how one of the more notorious pedophiles[1] was caught[2].
Re: It's all a blur
#59A simple solution is to use a system of linear equations.
Each row of a matrix is a linear equation, Ax = b
Each row contains kernel weightings A across the image X, B is the blurred pixel color.
The full matrix would be a terabyte, so take advantage of the zeros and use an efficient solve for X instead of inversion.
Enhance really refers to combining multiple images. (stacking) Each pixel in a low res image was a kernel over the same high res image. So undoing a 100 pixel blur is equivalent to combining 10,000 images for 100x super resolution.
Re: It's all a blur
#60Sorry but this post is the blind leading the blind, pun intended. Allow me to explain, I have a DSP degree. The reason the filters used in the post are easily reversible is because none of them are binomial (i.e. the discrete equivalent of a gaussian blur). A binomial blur uses the coefficients of a row of Pascal's triangle, and thus is what you get when you repeatedly average each pixel with its neighbor (in 1D). Wh…
If you have an endless pattern of ..., -1, 1, -1, 1, -1, 1, ... and run box blur with a window of 2 or 4, you get ..., 0, 0, 0, 0, 0, 0, ... too.
Other than that, you're not wrong about theoretical Gaussian filters with infinite windows over infinite data, but this has little to do with the scenario in the article. That's about the information that leaks when you have a finite window with a discrete step and start at a well-defined boundary.