Wouldn't it be really neat to apply this to HD movie sequences, and hear what the sounds on the set and the voices of actors were like pre-production? And how unreal some of the sounds must have turned out with all the visual tweaking that happens in production?
You missed the part where the video's frames per second has to be higher than the audio frequency.
Extracting audio from visual information
61–70 of 133 posts
Re: Extracting audio from visual information
#62Wouldn't it be really neat to apply this to HD movie sequences, and hear what the sounds on the set and the voices of actors were like pre-production? And how unreal some of the sounds must have turned out with all the visual tweaking that happens in production?
You missed the part where the video's frames per second has to be higher than the audio frequency.
Re: Extracting audio from visual information
#63It doesn't seem to mention what kind of camera you'd need to do this at the mentioned distance (15 feet). I'm assuming it's at least a couple thousand bucks, or maybe even so expensive that most professional photographers won't have it, but does anyone actually know? Or did I miss it in the article?
There are also some consumer cameras that can capture almost 1,000 fps from a small sensor window. I wonder if those would work.
Re: Extracting audio from visual information
#64Re: Extracting audio from visual information
#65This is amazing. Imagine taking high definition video of a crowd of people. You could pick out objects nearby and hear what individuals are saying. You could nearly produce a 3D auditorium by sampling different points in a video. Couple this with a 3D camera and an Oculus Rift, you could have something incredible.
http://en.wikipedia.org/wiki/Cocktail_party_effect
http://en.wikipedia.org/wiki/Independent_component_analysis
Edit: Spacing.
Re: Extracting audio from visual information
#66It doesn't seem to mention what kind of camera you'd need to do this at the mentioned distance (15 feet). I'm assuming it's at least a couple thousand bucks, or maybe even so expensive that most professional photographers won't have it, but does anyone actually know? Or did I miss it in the article?
Re: Extracting audio from visual information
#67Re: Extracting audio from visual information
#68Re: Extracting audio from visual information
#69This is from the movie Eagle Eye, right? The evil computer watches the vibrations in a cup of coffee while someone is speaking.
Re: Extracting audio from visual information
#70Earlier quoted context omitted.
You missed the part where the video's frames per second has to be higher than the audio frequency.
Some cheap video cameras don't shutter the frames (grabbing all pixels at once), they progress thru pixels - leading to a usually-annoying stretchy visual effect. If the sound affects much of the frame with suitable uniformity, you've turned a 60fps camera into a 2MHz "visual" audio sampler (lots of noise, artifacts, & other difficulties aside).