Algorithm recovers speech from a potato-chip bag filmed through glass (2014) • Hacker News Board

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#61

Post by baddox » Mon, Aug 06, 2018, 12:40 AM UTC

fipple wrote: post #60

jwilk wrote: post #45

Earlier quoted context omitted.

> [Jones] claimed to have extracted the hum of the potter's wheel from the grooves of a pot, So far so good... > and the word "blue" from an analysis of patch of blue color in a painting. What the hell?

I don’t know about the techniques but the theory isn’t prima facie impossible. A paintbrush can act like a microphone just like anything else, and if the paintbrush (more likely a putty knife or more rigid object) picked up a sound while applying paint, that could manifest in the paint layer.

And...someone was saying “blue” the instant they applied the blue paint? That’s certainly not impossible to believe, it just seems like a bit of a stretch.

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#62

Post by namibj » Mon, Aug 06, 2018, 12:47 AM UTC

Sharlin wrote: post #38

zeta0134 wrote: post #29

Earlier quoted context omitted.

It's known as the Rolling Shutter effect. It's most commonly observed when attempting to take still imagery of very fast moving objects like rotors on airplanes, but affects pretty much any image sensor that isn't designed specifically to work around it. The following Wikipedia article goes into far more depth, but the basic problem is that an image sensor doesn't sample every pixel instantly, but instead reads them…

And it is why mechanical shutters are still a thing in dedicated cameras. It is still much faster to move two pieces of metal or carbon fiber over the sensor than it is to read out a multi-megapixel CMOS sensor. When shooting video mechanical shutters cannot be used so the rolling shutter effect can cause artifacts such as fast-moving subjects appear slanted.

Err, modern global shutters like those made by CMOSIS/ams are _much_ faster than that.

Their fast one is an APS-C sensor with 4k*3k Px, a shutter closing time of 1s/~120000 and a minimum shutter open time of ~1s/50000. The shutter closing time might be even faster, just reconstruction from frame overhead time values I remember, and adjusting for the share the row skew had. Check the datasheet if you like to.

If you know a mechanical shutter that can do such, I'd like to know.

This, by chance, allows laser-flash illumination of objects that are behind a close wall of fog, as you can keep the shutter closed while the light travels to the further away object of interest. You will have the blur, but no longer the massive constrast loss due to light pollution. If it's not as bad as fog, and just e.g. normal rainfall or such, you lack the reflection artifacts that would be common, and only retain the refraction artifacts from the light passing through it by necessity.

These sensors do lack a little dynamic range, but you can compensate with some slight trickery, see the datasheet, to get ~15 stops out of this particular device.

Rolling shutter has applications, but videos for users of low knowledge and high ambitions is not one of them.

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#63

Post by jetrink » Mon, Aug 06, 2018, 12:50 AM UTC

baddox wrote: post #61

fipple wrote: post #60

Earlier quoted context omitted.

I don’t know about the techniques but the theory isn’t prima facie impossible. A paintbrush can act like a microphone just like anything else, and if the paintbrush (more likely a putty knife or more rigid object) picked up a sound while applying paint, that could manifest in the paint layer.

And...someone was saying “blue” the instant they applied the blue paint? That’s certainly not impossible to believe, it just seems like a bit of a stretch.

Maybe it was painted by Bob Ross?

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#64

Post by angry_octet » Mon, Aug 06, 2018, 1:18 AM UTC

Digit-Al wrote: post #20

Forgive me, as someone with zero actual knowledge in this field. This may be a naive question, but wouldn't this be fairly easily defeated by playing songs in the background?

You could identify the song and then subtract it. If you add which noise it raises the noise floor, but won't effect overall decoding that much, because there is such deep knowledge of which phonemes are likely to follow in a given sequence. You have to assume there is a model trained for each participant, e.g. using telephone intercepts, other listening devices.

Maybe if you simultaneously played back segments of dozens of conversations of the participants talking. That would certainly be confusing for the participants.

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#65

Post by jdietrich » Mon, Aug 06, 2018, 1:19 AM UTC

sbierwagen wrote: post #54

jdietrich wrote: post #40

Earlier quoted context omitted.

>the UK and Chinese CCTV networks comes to mind Minor nitpick, but the UK doesn't have a CCTV network. It has a huge number of privately owned CCTV cameras and a relatively small number of CCTV cameras operated by individual local authorities and police forces. The privately-owned cameras aren't joined up in any useful way and are often of very poor quality; the publicly owned cameras are overwhelmingly used for real…

Run the algo on any reflective objects in the field of view, feed the outputted audio to a transcription NN. Save all high confidence transcribed voice as compressed txt. Save the source video if you pass a threshold of danger keywords. ("bomb", "kill", "prime minister", "didn't pay my tv licence")

Best case scenario, you'll need a nuclear power station and a few dozen data centers full of ASICs. Oh, and an internet's worth of extra bandwidth.

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#66

Post by angry_octet » Mon, Aug 06, 2018, 1:35 AM UTC

lifeisstillgood wrote: post #47

devoply wrote: post #37

Earlier quoted context omitted.

Some of them do, there are geosynchronous spy satellites. https://www.popsci.com/gaofen-4-worlds-most-powerful-geo-spy... "It may also have a lower resolution video streaming capacity."

Ok I was coming to say "spy satellites at 40,000 km up - I doubt they can see anything. And if the linked article is correct the Chinese satellites up there have a resolution of 50m - good luck finding a crisp packet. But the new generation "might" have a resolution of 1m. which is insane. Then again, good luck knowing which square meter of the 1/3 of the earths surface you can see, has the crisp packet in. I still t…

Recent Chinese optical satellites are thought to have 10cm resolution. All these are low earth orbit. Depends on cloud cover, atmospheric turbulence and look angle.

Still impractical to get sound vibrations from that. But a drone with a laser would work for windows. Think listening in on a conversation in a car.

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#67

Post by xapata » Mon, Aug 06, 2018, 2:16 AM UTC

wiz21c wrote: post #50

pushing the argument to its limit : what happens if we can recover any speech from this or that material (moving leaves, bags, lips movements (AI would be good at that), changes in magnetic fields whatever). should we assume that there is no such thing as a "safe place" anymore ? One could argue there never was (because, for example, police can eavesdrop) but, this article implies it can reach a whole new level... Ho…

Microphones are already quite good. If you can be seen, you can be "heard". That has been true for many years. I think there was a scene in the movie Enemy of the State involving cone mics. The difference with this technology is that someone can "hear" you even if you're around a corner.

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#68

Post by ChuckMcM » Mon, Aug 06, 2018, 2:47 AM UTC

Digit-Al wrote: post #20

Forgive me, as someone with zero actual knowledge in this field. This may be a naive question, but wouldn't this be fairly easily defeated by playing songs in the background?

Yes, or even better moderate white noise. Ambient sound will lower the signal to noise ratio between ambient audio and the audio of interest and so essentially render using this to eavesdrop on someone effectively moot.

As an exercise in using tertiary effects to pull in signals you might otherwise be prevented from receiving, well that is pretty cool.

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#69

Post by ianai » Mon, Aug 06, 2018, 3:11 AM UTC

baddox wrote: post #61

fipple wrote: post #60

Earlier quoted context omitted.

I don’t know about the techniques but the theory isn’t prima facie impossible. A paintbrush can act like a microphone just like anything else, and if the paintbrush (more likely a putty knife or more rigid object) picked up a sound while applying paint, that could manifest in the paint layer.

And...someone was saying “blue” the instant they applied the blue paint? That’s certainly not impossible to believe, it just seems like a bit of a stretch.

It’d be a stretch if they were talking about ancient aliens. Saying the color they’re painting isn’t a change of context.

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#70

Post by ianai » Mon, Aug 06, 2018, 3:16 AM UTC

eganist wrote: post #44

MrQuincle wrote: post #33

Earlier quoted context omitted.

I also once thought of that, but maybe there are other ways in which large vibrations have left fingerprints on materials. I don't mean gravitational waves, but really acoustic phenomena. Like a comet impact or a vulcano outburst. What's a delicate material which would be able to record sound but that doesn't get destroyed by time? Clay that dries up is a logical one.

Old glass or porcelain might be the most viable candidates for a first look as they would be cooling off from a heated, more malleable state, potentially capturing sound from the immediate vicinity as they cooled off. I have no science to back this up. It's just a hunch.

That may be complicated from old glass never fully hardening. But it sounds plausible enough.

Possibly mortar in walls could record the workers taking. All sorts of pottery start out malleable, so they might be candidates. Cave paintings might be an option - wonder whether being finger painted would leave biological fingerprints behind - heart rate, for instance.

Pretty wild idea. Here’s hoping it has some legs.

Maybe someday we’ll learn about places like Stonehenge this way.