Live data from Hacker News

Video Stabilization on YouTube

googleresearch.blogspot.in

31–40 of 51 posts

Re: Video Stabilization on YouTube

#31
post #15

I wonder when will we get this quality http://pages.cs.wisc.edu/~fliu/project/3dstab.htm

Thanks for that link, this should have a submission of its own. I can easily see how something like this would make a 'point and shoot' video camera really useful. Think "Flip Camera meets James Cameron"

Re: Video Stabilization on YouTube

#32
post #9

They should apply this during movie fighting scenes. Then we might actually see the fight instead of the blur caused by "exciting cameras".

Higher frame rates would help a lot with the blur. I think Peter Jackson made a mistake shooting The Hobbit at 48 FPS for the entire movie. He should have shot most of it at the traditional 24 FPS but used 48 or 72 for fast motion shots. Hopefully his blunder won't poison high FPS forever in the minds of filmgoers.

Re: Video Stabilization on YouTube

#33
post #18

Seems to work well- I just tried it on a tiger cub vid, which I've had for years and was taken with an el-cheapo digital camera: Shaky version: http://youtu.be/g5P9WwHdOSI Stabilised version http://youtu.be/wXfbRUk_1Bg There is some motion blur (around 24 seconds, when the momma tiger lies down) which is a bit puzzling once the image has been stabilised. But hey, it's better than it was before.

It also looks like the stabilized video is magnified a little bit. Regardless, these tools are really neat and they are still in their infancy. It's good to see that GooTube still believes in user-generated content.

Cropping and magnifying the video is required for all methods of video stabilization, because otherwise you would see the edge of the original frame move in and out of view.

Re: Video Stabilization on YouTube

#34
post #15

I wonder when will we get this quality http://pages.cs.wisc.edu/~fliu/project/3dstab.htm

Have you tried it? I have and I'd say the quality is pretty close if not the same. The much bigger problem to solve now is that shaky videos shot in less-than-perfect lighting contain motion blur, which is extremely hard to remove. You'll notice that all of these demo videos were conveniently shot outside in direct sunlight and contain no motion blur at all.

Re: Video Stabilization on YouTube

#37
post #28

Earlier quoted context omitted.

Probably a staged setup, inducing a regular shaking pattern, but I think generating rolling shutter algorithmicly would be a stretch for them, so I call physical but maybe voluntary shake.

Or maybe it's due to the 'rolling' shutter causing shear stretching as mentioned in the article

But the point is that if they faked shaking algorithmically then the original signal would be shear-free right ?

Re: Video Stabilization on YouTube

#38
Isn't quality inherently lost because the same video has to be reencoded again but without the shakes? Also, I just tested on a video and it looked slightly smudgy. OK, so if I am filming driving down a dirt road or after half a bottle of Jack Daniel's (or both) then it'd be good, otherwise it does more harm than good.

Re: Video Stabilization on YouTube

#39
post #22

Earlier quoted context omitted.

This is amazing. Wonder if there is a commercial implementation yet?

These guys do some pretty amazing stuff, video stabilization is one of the applications: http://www.2d3.com/

Exactly, I also though of 2d3 as soon as I have seen that blog post. I came to know the company since it is linked by the (very valuable, if I may) book "Multiple View Geometry in Computer Vision", http://www.amazon.com/Multiple-View-Geometry-Computer-Vision... . IIRC the company is run by Andrew Fitzgibbon http://research.microsoft.com/en-us/um/people/awf/ , which now seems odd to me since he's a full time researcher at Microsoft. I wander how Andrew's algorithm in the 2d3.com product differs from the one published by the youtube guys in the post.

Re: Video Stabilization on YouTube

#40
As mentioned in the blogpost, the rolling-shutter version of this won the best paper prize at the International Conference on Computation Photography (ICCP), which was held last weekend in Seattle. This is a fairly new but very high quality conference. In many respects, I prefer it to the standard-bearing vision conferences like CVPR, ICCV, or ECCV -- although of course, ICCP is more narrowly focused on computational imaging and photography applications.

In their talk, the authors of this work showed many more video results and they were all quite impressive. In fact, they were good enough to fall into an "uncanny valley of motion", similar to the "uncanny valley" of faces or humans [1] that most people are familiar with. I.e., the motion correction was almost perfect, but just enough off that something felt vaguely surreal about the results. Nevertheless, it's a nice step forward.

Also, as others have pointed out, this is a fully uncalibrated method -- requiring no knowledge of how the video was captured. If you do have some knowledge, then you can often exploit it to do better. But the authors mentioned that most videos uploaded to youtube have either no calibration information, or if present, it's often incorrect. As such, it made sense for them to focus on the uncalibrated case.

Finally, I should point out that rolling shutter, standard on most mobile cameras, is causing all sorts of problems for traditional image and video analysis algorithms, which often make the assumption, sometimes implicitly, that the entire frame was captured at a single instance in time. This is not true anymore, and can lead to gross errors in many methods. Hence the many recent papers on correcting for, and in some cases exploiting, rolling shutter effects.

A recent interesting work along these lines from my former lab at Columbia University is "coded rolling shutter photography: flexible space-time photography" [2]. This paper takes advantage of the fact that different rows in an image are seeing the world at slightly different instances in time to do things like high-speed photography, HDR imaging, etc.

[1] http://en.wikipedia.org/wiki/Uncanny_valley

[2] http://www.cs.columbia.edu/CAVE/projects/crsp/

Post reply on HN