Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Recovering redacted information from pixelated videos

positive.security

Post reply on HN
Print view View original

61–70 of 70 posts

Re: Recovering redacted information from pixelated videos

#61

Post by MauranKilom »

abbeyj wrote: post #57
MauranKilom wrote: post #46

Earlier quoted context omitted.

If you can figure out, for each frame, which sets of (pre-aligned) pixels have been averaged, you can create a large system of equations that captures those relations and solve it to find the unblurred pixel values. Depending on camera movement (and whether you might get "ground truth" information from pixels entering and leaving the areas near the borders) the system will be more or less well-conditioned. I'm going…

There are some (fairly old) papers that might contain some useful ideas for you: - http://www.eyetap.org/papers/docs/mann94virtual.pdf - http://wearcam.org/orbits/index.html I seem to recall that there used to be a video showing this approach in action. As input it took a video panning across a shelf full of books where the resolution was so low that the titles were illegible. And as output it produced a video with h…

Yes, it all boils down to point spread functions. In the mosaic case, the PSF varies locally (per pixel) and temporally (in different video frames). The paper you link similarly details how they figure out the PSF. You can theoretically also do the entire thing without knowing the PSF, which is called blind deconvolution: https://en.wikipedia.org/wiki/Blind_deconvolution

Re: Recovering redacted information from pixelated videos

#62

Post by MauranKilom »

breakingsystems wrote: post #36
MauranKilom wrote: post #31

Earlier quoted context omitted.

Sure, but the technique used was also very trivial. Just aligning and averaging all the video frames basically leaves a mosaic-pixel-sized blur on everything (assuming the camera movement is uncorrelated with the mosaic grid). You can get much further by applying deconvolutions and using more math. I've been meaning to put some time into this myself but never got it off the ground. I wonder if the author would be ope…

> I wonder if the author would be open to making e.g. the car data available? Here you go: - The stabilized frames out of blender (with 4 blurry frames in a separate folder): https://breaking.systems/plate_frames_sorted.zip - The original video in case you'd like to improve the stabilization as well: https://breaking.systems/plate_vid_orig.mp4 Would love to hear back in case you'll tackle it!

First challenge is going to be figuring out the grid alignment in the stabilized frames. But I have a decent idea how to tackle that, which I'll hopefully get to tomorrow!

Re: Recovering redacted information from pixelated videos

#63

Post by MarkLowenstein »

RunSet wrote: post #4

Yet again mosaic blur's inferiority to opaque rectangle is demonstrated.

Have you ever hand-written a word but want to hide it? One cross-out line doesn't do the trick. In fact, a full minute of trying to hide it with an ugly darkened-in box usually doesn't even do it. But if you just write a couple random letters over each existing one, people have basically no ability to recognize your original word.

That is my one insight. Take it for what it's worth.

Re: Recovering redacted information from pixelated videos

#65

Post by rasz »

Syzygies wrote: post #55
throwawayciv651 wrote: post #38

Earlier quoted context omitted.

Not off the shelf but here are some tools. I have no experience with them. Wolf binarization - I think it makes the text more clear before OCR. https://github.com/chriswolfvision/local_adaptive_binarizati... This thing OCRs the pdf using Tesseract OCR https://github.com/ocrmypdf/OCRmyPDF/ Two other pdf tools https://github.com/qpdf/qpdf https://github.com/pikepdf/pikepdf

I'll play with the Wolf filter, thanks. Math typesetting is too messy for current OCR tools. It would be nice to reverse-engineer the LaTeX source for a math paper, but not likely soon. OCR for the language would help in mind-mapping a web connecting my saved papers, but I wouldn't use it for reading. I want everything to look like a 600dpi scan mixed down, as I would make, rather than what the libraries thought woul…

I imagine you could go really far with a dumb approach or matching against pre existing font database.

Re: Recovering redacted information from pixelated videos

#66

Post by mleonhard »

It seems that we could increase camera resolution by putting the sensor on a vibrating platform, capturing a stream of frames, and processing them into a single image. The paper mentions Google camera software doing this with hand tremor. Is there any instance of intentionally shaking a camera to increase resolution like this?

I predict that future super high-resolution camera rigs will be whirling contraptions, spinning in 3 dimensions to improve 3D resolution. And the best still camera will be a wand (linear sensor array) on an articulated head that moves like a chicken's head, capturing during movement. The sound of a camera will be whoosh instead of click.

Re: Recovering redacted information from pixelated videos

#67

Post by brickers »

mleonhard wrote: post #66

It seems that we could increase camera resolution by putting the sensor on a vibrating platform, capturing a stream of frames, and processing them into a single image. The paper mentions Google camera software doing this with hand tremor. Is there any instance of intentionally shaking a camera to increase resolution like this? I predict that future super high-resolution camera rigs will be whirling contraptions, spin…

I believe Pentax have a system like this in some DSLR models

Re: Recovering redacted information from pixelated videos

#68

Post by MauranKilom »

scoot wrote: post #59
MauranKilom wrote: post #31

Earlier quoted context omitted.

Sure, but the technique used was also very trivial. Just aligning and averaging all the video frames basically leaves a mosaic-pixel-sized blur on everything (assuming the camera movement is uncorrelated with the mosaic grid). You can get much further by applying deconvolutions and using more math. I've been meaning to put some time into this myself but never got it off the ground. I wonder if the author would be ope…

> I wonder if the author would be open to making e.g. the car data available? Interesting - this is same incorrect use of e.g. that the author made in a couple of places. Contrary to (apparently popular) belief, "i.e." and "e.g." can't simply be used as direct replacements for their English equivalents. "e.g." is used to introduce one or more examples that satisfy a previously provided general form, for example: I pr…

Interesting, thanks for bringing this to my attention! Do you have any reference that explains this rule? I noticed on Wikipedia that introducing multiple examples used to actually have a different abbreviation (ee.g. or ee.gg.), so clearly something is already lost in translation here...

In any case, what I wrote is really just a shorthand for more cumbersome formulations (like "...open to making your data, e.g. the car [data], available?" - that would hopefully be correct?), and reducing text is the whole point of using an abbreviation in the first place. But I'm open to striving for more consistent usage, so if you can refer me to some kind of authority on how to mix Latin abbreviations with English text, I'd be curious about it!

Re: Recovering redacted information from pixelated videos

#69

Post by HanClinto »

MauranKilom wrote: post #62
breakingsystems wrote: post #36

Earlier quoted context omitted.

> I wonder if the author would be open to making e.g. the car data available? Here you go: - The stabilized frames out of blender (with 4 blurry frames in a separate folder): https://breaking.systems/plate_frames_sorted.zip - The original video in case you'd like to improve the stabilization as well: https://breaking.systems/plate_vid_orig.mp4 Would love to hear back in case you'll tackle it!

First challenge is going to be figuring out the grid alignment in the stabilized frames. But I have a decent idea how to tackle that, which I'll hopefully get to tomorrow!

I'm interested in following along with your progress!

Re: Recovering redacted information from pixelated videos

#70

Post by godelski »

Pxtl wrote: post #3

The article references the famous face-depixellating PULSE GAN. Let's remember that PULSE also gave us Barry O'Bama. https://www.google.com/amp/s/www.theverge.com/platform/amp/2...

I'm not sure why PULSE is being called out here. This is extremely common on any super-resolution trained on extremely biased datasets like FFHQ. In fact, this discussion is something that led to the dataset being changed. The authors actually extended their research to discuss the bias[0]. I'll also link to the Reddit discussion[1]. IMO the authors here responded to this correctly. It is important to remember that algorithms are only good on their in distribution datasets. And kudos on the authors for doing more experiments and including work on a less racially and sexually biased dataset.

[0] section 6 https://arxiv.org/pdf/2003.03808.pdf

[1] https://www.reddit.com/r/MachineLearning/comments/hk2ryn/d_h...

Post reply on HN