The Visual Microphone: Passive Recovery of Sound from Video
people.csail.mit.edu
The Visual Microphone: Passive Recovery of Sound from Video
1–10 of 41 posts
Re: The Visual Microphone: Passive Recovery of Sound from Video
#2Previous discussion: https://news.ycombinator.com/item?id=8131785
Re: The Visual Microphone: Passive Recovery of Sound from Video
#3Previous discussion: https://news.ycombinator.com/item?id=8131785
I'll partially copy two interesting comments from that thread:
> [A video with] 60 frames per second only allowed to identify the speaker and the number of people in the room.
> The demo in the video is based on a high-speed (1000+fps) recording by a special camera, not on 'normal' video.
Re: The Visual Microphone: Passive Recovery of Sound from Video
#4[deleted]
Re: The Visual Microphone: Passive Recovery of Sound from Video
#5They're making progress. At 5000 FPS, it's not surprising that they can recover audio. But from 60 FPS, that's striking. That works because some imagers don't take the whole frame at once.
Re: The Visual Microphone: Passive Recovery of Sound from Video
#6Previous discussion: https://news.ycombinator.com/item?id=8131785
I'll partially copy two interesting comments from that thread: > [A video with] 60 frames per second only allowed to identify the speaker and the number of people in the room. > The demo in the video is based on a high-speed (1000+fps) recording by a special camera, not on 'normal' video.
A standard mobile phone today can do 240fps. We're only a few years away from 1000+ for in everyday devices.
Re: The Visual Microphone: Passive Recovery of Sound from Video
#7Earlier quoted context omitted.
I'll partially copy two interesting comments from that thread: > [A video with] 60 frames per second only allowed to identify the speaker and the number of people in the room. > The demo in the video is based on a high-speed (1000+fps) recording by a special camera, not on 'normal' video.
A standard mobile phone today can do 240fps. We're only a few years away from 1000+ for in everyday devices.
Really? Isn't speed limited by the amount of light that can be received in a small lens like on a phone?
Re: The Visual Microphone: Passive Recovery of Sound from Video
#8This is an interesting extension of the ideas from this paper by the same author/s http://people.csail.mit.edu/mrub/vidmag/.
Re: The Visual Microphone: Passive Recovery of Sound from Video
#9Earlier quoted context omitted.
A standard mobile phone today can do 240fps. We're only a few years away from 1000+ for in everyday devices.
Really? Isn't speed limited by the amount of light that can be received in a small lens like on a phone?
>Isn't speed limited by the amount of light that can be received in a small lens like on a phone?
It's also limited by sensor noise and efficiency.
Presumably, we'll push all three of these limits over the next few years.
Re: The Visual Microphone: Passive Recovery of Sound from Video
#10They're making progress. At 5000 FPS, it's not surprising that they can recover audio. But from 60 FPS, that's striking. That works because some imagers don't take the whole frame at once.
Almost all consumer cameras have rolling shutters. In fact for this experiment, the crappier the camera, the better, as it's less pronounced in a lot of higher-end cameras. I'd suspect they might even be able to do better sound recovery with a GoPro than a DSLR.