Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

news.mit.edu

Post reply on HN
Print view View original

31–40 of 111 posts

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#31

Post by fooker »

freewilly1040 wrote: post #9
fooker wrote: post #8

Earlier quoted context omitted.

>Thinking atomic bombs :( The same ones which have prevented large scale open military conflicts involving superpowers for the last 60+ years? Atom bombs have likely saved more lives than they have taken, if we had conventional wars with modern technology without MAD.

That’s one read on history. Another is that the world has been tremendously lucky to not have suffered a nuclear holocaust since WWII. Many powerful people have done everything they could to advocate for nuclear bombings in the past 60 years, and we’ve had some extremely close calls with near accidental launches.

You are right. It might be too soon to definitively say which take is correct. It might be the case that a disaster (natural, economic, alien invasion, etc) might spark yet another world war.

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#32

Post by kryogen1c »

PowerfulWizard wrote: post #19
PeterisP wrote: post #14

Earlier quoted context omitted.

"Will you only speak of sensitive subjects in rooms with no windows" - yes, the building standards for secure facilities (i.e. a room where discussing classified information above a certain grade is permitted) mandate no external windows, and much more than that.

I tried to look this up, info here: https://www.dni.gov/files/NCSC/documents/Regulations/Technic... (Ch. 3 sec. F, pg. 13) F. SCIF Window Criteria 1. Every effort should be made to minimize or eliminate windows in the SCIF, especially on the ground floor. 2. Windows shall be non-opening. 3. Windows shall be protected by security alarms in accordance with Chapter 7 when they are within 18 feet of the ground or an acce…

Beat me to it.

https://en.wikipedia.org/wiki/Sensitive_Compartmented_Inform...

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#33

Post by MrQuincle »

cma wrote: post #15

At some point it is (controversially) hypothesized that we may be able to pull imprinted recordings off of ancient artifacts: https://en.wikipedia.org/wiki/Archaeoacoustics#Past_interpre...

I also once thought of that, but maybe there are other ways in which large vibrations have left fingerprints on materials. I don't mean gravitational waves, but really acoustic phenomena. Like a comet impact or a vulcano outburst. What's a delicate material which would be able to record sound but that doesn't get destroyed by time? Clay that dries up is a logical one.

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#34

Post by zeta0134 »

maxerickson wrote: post #26
toomuchtodo wrote: post #5

Earlier quoted context omitted.

With enough processing power, you can listen to anyone as long as you can get an object in frame that’s vibrating with speech (the UK and Chinese CCTV networks comes to mind). Depending on resolution, I’d expect this to be the case for any footage already stored. It’s just a matter of having a distributed computing job kicked off to comb through video data and add the additional audio metadata. Will you only speak of…

Don't sweat the CCTV or recordings: Reconstructing audio from video requires that the frequency of the video samples — the number of frames of video captured per second — be higher than the frequency of the audio signal. In some of their experiments, the researchers used a high-speed camera that captured 2,000 to 6,000 frames per second. That’s much faster than the 60 frames per second possible with some smartphones,…

Right, which is why the researchers were reliant on video artifacts caused by the rolling shutter effect. Even at NTSC resolution, that could get you 262.5 scanlines per frame at 60 FPS (525 at ~29.75 FPS, because of interlacing... NTSC is complicated), which could potentially get you ~15 KHz as a baseline sample rate, depending on how prominent the rolling shutter effect actually is for that image sensor, and how much of the frame your vibrating object consumes vertically. This isn't perfect and would require all sorts of calibration, but I could see it being used to recover windows of time at much higher sample rates, and make inferences about the inbetween data with some FFT analysis on what you recovered.

Basically, it's more the rate of _scanlines_ that matters for this technique, and the quality of the image sensor used. The rate of full frames isn't the limiter.

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#36

Post by perl4ever »

gh0zt wrote: post #4

“I’m sure there will be applications that nobody will expect. I think the hallmark of good science is when you do something just because it’s cool and then somebody turns around and uses it for something you never imagined. It’s really nice to have this type of creative stuff.” Thinking atomic bombs :(

The US atomic bomb project didn't fit that description, I don't think. It was started expressly because people believed they were in a race with the Axis to develop one first. I am surprised that Einstein's letter to FDR isn't all that universally known these days.[1]

[1]https://en.wikipedia.org/wiki/Einstein–Szilárd_letter

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#37

Post by devoply »

zerb wrote: post #11
devoply wrote: post #7

Now imagine doing this with a high res satellite anywhere on earth.

That would only be useful if everyone were outside, and you wanted a split second of many different people's conversations. Satellites move rather quickly relative to the Earth's surface.

Some of them do, there are geosynchronous spy satellites.

https://www.popsci.com/gaofen-4-worlds-most-powerful-geo-spy...

"It may also have a lower resolution video streaming capacity."

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#38

Post by Sharlin »

zeta0134 wrote: post #29
booleandilemma wrote: post #24

Because of a quirk in the design of most cameras’ sensors, the researchers were able to infer information about high-frequency vibrations even from video recorded at a standard 60 frames per second. That’s a pretty convenient quirk if you ask me.

It's known as the Rolling Shutter effect. It's most commonly observed when attempting to take still imagery of very fast moving objects like rotors on airplanes, but affects pretty much any image sensor that isn't designed specifically to work around it. The following Wikipedia article goes into far more depth, but the basic problem is that an image sensor doesn't sample every pixel instantly, but instead reads them…

And it is why mechanical shutters are still a thing in dedicated cameras. It is still much faster to move two pieces of metal or carbon fiber over the sensor than it is to read out a multi-megapixel CMOS sensor. When shooting video mechanical shutters cannot be used so the rolling shutter effect can cause artifacts such as fast-moving subjects appear slanted.

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#39

Post by kzrdude »

cma wrote: post #15

At some point it is (controversially) hypothesized that we may be able to pull imprinted recordings off of ancient artifacts: https://en.wikipedia.org/wiki/Archaeoacoustics#Past_interpre...

Well it's certainly thought provoking, thanks for the link.

Re: Algorithm recovers speech from a potato-chip bag filmed through glass (2014)

#40

Post by jdietrich »

toomuchtodo wrote: post #5
mirimir wrote: post #3

Rule of thumb is never having sensitive conversations where such exploits are feasible.

With enough processing power, you can listen to anyone as long as you can get an object in frame that’s vibrating with speech (the UK and Chinese CCTV networks comes to mind). Depending on resolution, I’d expect this to be the case for any footage already stored. It’s just a matter of having a distributed computing job kicked off to comb through video data and add the additional audio metadata. Will you only speak of…

>the UK and Chinese CCTV networks comes to mind

Minor nitpick, but the UK doesn't have a CCTV network. It has a huge number of privately owned CCTV cameras and a relatively small number of CCTV cameras operated by individual local authorities and police forces. The privately-owned cameras aren't joined up in any useful way and are often of very poor quality; the publicly owned cameras are overwhelmingly used for real-time monitoring of busy city centre locations.

Installing CCTV cameras is cheap and easy, but usefully monitoring them is expensive and difficult, even with whizz-bang CV algorithms. I'm deeply sceptical as to how useful any state-level CCTV network would be for mass surveillance. 20 million 4K/30fps cameras would produce something in the region of two exabytes per day; just storing that data would cost about $7bn per month.

Post reply on HN