Forgive me, as someone with zero actual knowledge in this field. This may be a naive question, but wouldn't this be fairly easily defeated by playing songs in the background?
I think that would just add music to the sound recovered.
Algorithm recovers speech from a potato-chip bag filmed through glass (2014)
21–30 of 111 posts
Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)
#22Forgive me, as someone with zero actual knowledge in this field. This may be a naive question, but wouldn't this be fairly easily defeated by playing songs in the background?
A fan would probably work too, as it adds more white noise to the already noisy signal
Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)
#23“I’m sure there will be applications that nobody will expect. I think the hallmark of good science is when you do something just because it’s cool and then somebody turns around and uses it for something you never imagined. It’s really nice to have this type of creative stuff.” Thinking atomic bombs :(
That's not really how the history of atomic bombs went down. Szilàrd imagined the destructive power of chain reactions around 1932, nuclear fission was discovered in 1938, and soon after the Manhattan project was started with the clear intention of understanding the science enough to make a bomb.
I frequently encounter people who believe that nuclear energy was harnessed initially for power generation and then co-opted for destructive purposes. The first nuclear reactor was created to enrich uranium to make a bomb.
Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)
#24Because of a quirk in the design of most cameras’ sensors, the researchers were able to infer information about high-frequency vibrations even from video recorded at a standard 60 frames per second.
That’s a pretty convenient quirk if you ask me.
Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)
#25Forgive me, as someone with zero actual knowledge in this field. This may be a naive question, but wouldn't this be fairly easily defeated by playing songs in the background?
It might be possible to cancel that noise out. But something like a pink noise generator or an office grey noise generator would probably fool it, the same way it would fool a microphone at distance
Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)
#26Rule of thumb is never having sensitive conversations where such exploits are feasible.
With enough processing power, you can listen to anyone as long as you can get an object in frame that’s vibrating with speech (the UK and Chinese CCTV networks comes to mind). Depending on resolution, I’d expect this to be the case for any footage already stored. It’s just a matter of having a distributed computing job kicked off to comb through video data and add the additional audio metadata. Will you only speak of…
Don't sweat the CCTV or recordings:
Reconstructing audio from video requires that the frequency of the video samples — the number of frames of video captured per second — be higher than the frequency of the audio signal. In some of their experiments, the researchers used a high-speed camera that captured 2,000 to 6,000 frames per second. That’s much faster than the 60 frames per second possible with some smartphones, but well below the frame rates of the best commercial high-speed cameras, which can top 100,000 frames per second.
Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)
#27Forgive me, as someone with zero actual knowledge in this field. This may be a naive question, but wouldn't this be fairly easily defeated by playing songs in the background?
A fan would probably work too, as it adds more white noise to the already noisy signal
And might blow the chip bag away entirely :)
Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)
#28Now imagine doing this with a high res satellite anywhere on earth.
Satellites will never get such resolution due to optics and atmosphere. Drones however...
Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)
#29Because of a quirk in the design of most cameras’ sensors, the researchers were able to infer information about high-frequency vibrations even from video recorded at a standard 60 frames per second. That’s a pretty convenient quirk if you ask me.
It's known as the Rolling Shutter effect. It's most commonly observed when attempting to take still imagery of very fast moving objects like rotors on airplanes, but affects pretty much any image sensor that isn't designed specifically to work around it. The following Wikipedia article goes into far more depth, but the basic problem is that an image sensor doesn't sample every pixel instantly, but instead reads them out serially, and thus there's a tiny, tiny fraction of a second in-between each pixel being sampled. It's small enough to not matter for most purposes, but it's still there, and can be useful / detremental depending on the application.
Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)
#30Rule of thumb is never having sensitive conversations where such exploits are feasible.
With enough processing power, you can listen to anyone as long as you can get an object in frame that’s vibrating with speech (the UK and Chinese CCTV networks comes to mind). Depending on resolution, I’d expect this to be the case for any footage already stored. It’s just a matter of having a distributed computing job kicked off to comb through video data and add the additional audio metadata. Will you only speak of…
Back when this was new, I remember discussing it on Hackaday or somewhere. Someone remarked that they were amazed that the video could pick up on the motion at all, the video looked still to them. I responded that it really was still--the demo video had been lossfully encoded for the web (re-encoded by YouTube?), and the encoder took out the subtle changes.
I suspect that most footage already stored is similarly lossfully encoded, and that this technique isn't possible on it.