Live data from Hacker News

Extracting audio from visual information

newsoffice.mit.edu

121–130 of 133 posts

Re: Extracting audio from visual information

#121
post #80
post #59

Earlier quoted context omitted.

I had the exact same idea.

Why did I got down voted for that? It's true. Proof: http://www.crackajack.de/2014/08/04/visual-microphone-audio-...

Because posting "I had the exact same idea" without further interesting commentary is uninteresting.

Re: Extracting audio from visual information

#122

Earlier quoted context omitted.

lol that is the first thing that I thought as well. I just passed it off as a typical Hollywood trope, but now I'm quite delighted to see it become a reality.

> I just passed it off as a typical Hollywood trope I often wonder what makes people to pass off things like that as "typical tropes", where they are obviously realistic and doable.

"Any sufficiently advanced technology is indistinguishable from magic." Those who view it as magic will express it as a trope, even if it is merely advanced technology.

Just yesterday I chuckled when recalling a James Bond movie involving Bond driving a car in reverse by viewing a back-up camera. At the time, it formed an instant "trope" because it was so cool and novel. Yesterday, I was doing exactly that with the backup camera on my car - obviously realistic and doable technology.

Re: Extracting audio from visual information

#123

While it's cool to see this technique applied to consumer video, the general idea has been used in international espionage for a while now. In fact, Léon Theremin (yes, of the musical instrument) invented an early device capable of eavesdropping based on window vibrations: http://en.wikipedia.org/wiki/L%C3%A9on_Theremin#Espionage . I also recall that this is why all of the windows in the White House are fitted with t…

As a kid (in the early 80's!) I accidentally figured this out while "hacking" a sound based "walkie talkie" with a photo cell and a LED pointed at each other through a little telescope, from my house to my neighbors across the bock. The device worked, but had a bug. a hummmm on the receiver. For the longest time I could not figure out what was the "hummm", till I noticed it was there even with the transmitter off.. a…

>>till I noticed it was there even with the transmitter off.. and then, I heard strange voices even with the transmitter being off!

...and that is when little Johnny knew he had to lay off the shrooms.

:P

I love hearing stories like yours though. I, too, miss the child-like wonder of making discoveries. We take the physical world we live in for granted, but there's just so much out there to see and learn from.

Re: Extracting audio from visual information

#124
post #107

Wait, so what would be the best reflector for this type of thing? If I could host the ultimate cocktail party, placing ferns/etc all around the room, could I record every conversation with a single camera? What would those ferns look like? What would they be made of? I'm imagining christmas trees made of cellophane fibers or something.

If you're allowed to place the ferns, why not simply hide microphones in them?

I don't know, you may be very far away. The camera could be in space, or maybe on a plane. I mean, if you can hide microphones, the ferns would be irrelevant anyways, apart from being a good distraction.

Kind of off topic but just last night I was watching some (silent) lightning in the clouds, thinking how much more localized sound energy is when compared to light. In other words, I can see for light-years but can't hear much anything 100ft away. Or perhaps it's just our sensors are more sensitive to light waves than air pressure waves. Now I realize sound isn't so localized! It leaks. /rant

Re: Extracting audio from visual information

#125
post #39

Earlier quoted context omitted.

> Because of a quirk in the design of most cameras’ sensors, the researchers were able to infer information about high-frequency vibrations even from video recorded at a standard 60 frames per second.

The audio from the 60fps video sounds pretty bad though, which I suspect is mostly because of inherent maths/physics limitations rather than anything that software can improve. Edit: They mention capturing frequencies up to five times higher than the 60Hz frame rate, which would mean a maximum frequency of 300Hz, which would suggest the equivalent of 0.6kHz audio, which is a 73.5th of the audio rate of a CD. I doubt…

A CD has a lot of overkill for basic speech. Doing a test here with some speech samples, 2kHz is ugly but intelligible, 1kHz is a mess but mostly understandable with effort, and 600Hz is almost useless for trying to find words (without any practice or computer assistance, of course).

Re: Extracting audio from visual information

#126
post #79

Earlier quoted context omitted.

So maybe the only option would be The Hobbit?

The Hobbit still most likely doesn't use rolling shutter sensors, but I'm willing to be proven wrong.

Ah yes, that's probably true. I meant because of the higher frame rate.

Re: Extracting audio from visual information

#127
post #80

Earlier quoted context omitted.

Why did I got down voted for that? It's true. Proof: http://www.crackajack.de/2014/08/04/visual-microphone-audio-...

Because posting "I had the exact same idea" without further interesting commentary is uninteresting.

I only down vote when something is demonstrably wrong or offensive. I thought that's how everyone is doing it, so I wondered what could I have written that was wrong or offensive?

Re: Extracting audio from visual information

#128

Earlier quoted context omitted.

I record the dialog for your movies. Aprt from obvious sound effects like Darth Vader voices or so, actors' dialog is changed as little as possible during post production. It's not heavily EQed or comrpessed, because that would mean a corresponding change to the room tone and background noise, which would then fluctuate unnaturally as you went back and forth between the participants in a scene. In scenes with a lot o…

Thanks a lot for that insight. I will assume you work primarily for Hollywood with that description. It is interesting to hear about the behind-the-scenes from film industries in different parts of the world. In the Indian film industry (maybe not so much Bollywood, but more of the local ones like Tamil, Malayalam, etc) there's a LOT more post being done. Some entire Malayalam films have dubbed voices (for the origin…

Yeah, here in California we have the luxury to do good production sound almost all the time. Dubbing the voices later all sounds very different, because the recordings are not in the same space as the original production and so on. I forgot to think about that aspect, because here we always record production sound unless there is an impossibly high level of noise (eg from a wind machine or something). With modern cameras it's been very good for the sound department because they are so quiet, whereas older 16 and 35mm film cameras were quite noisy, like a sewing machine, and it required some work to prevent this messing up the dialog recording.

Re: Extracting audio from visual information

#129
post #42

Earlier quoted context omitted.

I know, but I still wonder about level of soundproof/mechanical vibration and the level of mic and camera sensor isolation VS level of optical vibration. In addition sound can travel many different paths since it is mechanical vibration, and traveling through sound proof glass is not the only path. Anw cool project.

I'm sure it's such a tiny amount that it isn't really relevant. Easy enough to test. Remove the bag of chips and just film the floor. See how much signal you can derive from that. Not exactly perfect science but gives a quick indication of "signal" that's reaching the system via other means.

Actually, what I think would be even better would be to record the bag of chips through the window, but play the sound outside where the camera is.

Re: Extracting audio from visual information

#130
post #127

Earlier quoted context omitted.

Because posting "I had the exact same idea" without further interesting commentary is uninteresting.

I only down vote when something is demonstrably wrong or offensive. I thought that's how everyone is doing it, so I wondered what could I have written that was wrong or offensive?

Downvotes also apply to "demonstrably content-free".

BTW: not everyone uses exactly your own personal guidelines.

BTW: HN Guidelines include "Resist complaining about being downmodded. It never does any good, and it makes boring reading."

Post reply on HN