Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Video Stabilization on YouTube

googleresearch.blogspot.in

Post reply on HN
Print view View original

41–50 of 51 posts

Re: Video Stabilization on YouTube

#41

Post by hamoid »

modeless wrote: post #34
hamoid wrote: post #15

I wonder when will we get this quality http://pages.cs.wisc.edu/~fliu/project/3dstab.htm

Have you tried it? I have and I'd say the quality is pretty close if not the same. The much bigger problem to solve now is that shaky videos shot in less-than-perfect lighting contain motion blur, which is extremely hard to remove. You'll notice that all of these demo videos were conveniently shot outside in direct sunlight and contain no motion blur at all.

I have only tried the one offered by YouTube, and not recently. I don't know if they have improved the algorithm in this respect, but for what I've seen in the past, the filter often creates a very eerie wobbly effect on the video, an effect that makes it look fake, like being underwater or drunk. It is slightly visible in the demo video if you observe the borders. This strange effect is totally absent on the link I posted, which I believe it's on a different level of quality. But I imagine it's computationally very expensive and can't be offered to millions of users for free.

Re: Video Stabilization on YouTube

#42

Post by chriszf »

hamoid wrote: post #15

I wonder when will we get this quality http://pages.cs.wisc.edu/~fliu/project/3dstab.htm

If you read the Google paper, you'll notice that they actually refer to this and other work by Liu et al. The overall technique is the same, estimate the original camera path, calculate an optimal camera path, retarget the input frames to a crop window that fits the optimal path.

The primary difference seems to be estimation and calculation technique. Liu's work does a structure-from-motion reconstruction, ie: rebuild a 3d model of the original scene. Google's work uses something called pyramidal Lucas-Kanade to do 'feature tracking' instead. This is sort of localized reconstruction, it seems to only care about the viewport differences from frame to frame. They then feed it through some linear programming voodoo to get the best path.

I don't understand either well enough to say why one is better than the other, although I'd guess it's because Lucas-Kanade is temporally and physically localized, it's easier to farm out to a parallel cluster than an SfM technique.

There also seems to be a difference on the rear end of the technique, having feature detection allows them to add 'saliency' constraints, ie: retarget based on the inclusion of certain features, like a person's face. Again, the math is beyond my understanding, but it seems like this isn't part of Liu's work.

Re: Video Stabilization on YouTube

#43

Post by modeless »

hamoid wrote: post #41
modeless wrote: post #34

Earlier quoted context omitted.

Have you tried it? I have and I'd say the quality is pretty close if not the same. The much bigger problem to solve now is that shaky videos shot in less-than-perfect lighting contain motion blur, which is extremely hard to remove. You'll notice that all of these demo videos were conveniently shot outside in direct sunlight and contain no motion blur at all.

I have only tried the one offered by YouTube, and not recently. I don't know if they have improved the algorithm in this respect, but for what I've seen in the past, the filter often creates a very eerie wobbly effect on the video, an effect that makes it look fake, like being underwater or drunk. It is slightly visible in the demo video if you observe the borders. This strange effect is totally absent on the link I…

An eerie wobbly effect is definitely visible to me in the videos from that paper.

Re: Video Stabilization on YouTube

#44

Post by hahainternet »

lith7 wrote: post #16

You can try something similar on linux using transcode with these two lines: transcode -J stabilize --mplayer_probe -i $infile transcode -J transform --mplayer_probe -i $infile -y xvid4 -o $outfile I found this blog post with more info: http://kevin.deldycke.com/tag/transcode/

Just as a note for anyone using this, you'll want to read some of the options as well. The amount of smoothing is hard to get right automatically, and you will want different values for different effects.

Having said that, my results with this tool have been excellent in the past.

Re: Video Stabilization on YouTube

#45

Post by baddox »

objclxt wrote: post #20
nextparadigms wrote: post #13

I'd like to see this come in Android so you can automatically shoot and record videos without the shaking in them. It would be a great selling point in my opinion.

You should already start to see that working its way out: the iPhone 4S does video stabilisation, I am sure high-end Android phones do or will start to do the same. The algorithm being discussed here is specifically designed for when information about the camera or environment is not available: there are much better ways of carrying out digital image stabilisation on the device itself, such as using the accelerometer…

The iPhone 4S algorithm is far less aggressive than YouTube's.

Re: Video Stabilization on YouTube

#46

Post by est »

Too wrote: post #17

What's the next step? Include accelerometer data from the camera, synced with the video, to use as support for stabilizing algorithms.

that's a really good idea actually. Maybe the video container format could add that metadata support.

Re: Video Stabilization on YouTube

#47

Post by elithrar »

apu wrote: post #40

As mentioned in the blogpost, the rolling-shutter version of this won the best paper prize at the International Conference on Computation Photography (ICCP), which was held last weekend in Seattle. This is a fairly new but very high quality conference. In many respects, I prefer it to the standard-bearing vision conferences like CVPR, ICCV, or ECCV -- although of course, ICCP is more narrowly focused on computational…

> Finally, I should point out that rolling shutter, standard on most mobile cameras, is causing all sorts of problems for traditional image and video analysis algorithms, which often make the assumption, sometimes implicitly, that the entire frame was captured at a single instance in time. This is not true anymore, and can lead to gross errors in many methods. Hence the many recent papers on correcting for, and in some cases exploiting, rolling shutter effects.

This is not just true of mobile phones, but of any current CMOS-sensored imaging device (most of them on the market). Compact cameras and SLR's included.

Re: Video Stabilization on YouTube

#48

Post by elithrar »

modeless wrote: post #32
Pelayo wrote: post #9

They should apply this during movie fighting scenes. Then we might actually see the fight instead of the blur caused by "exciting cameras".

Higher frame rates would help a lot with the blur. I think Peter Jackson made a mistake shooting The Hobbit at 48 FPS for the entire movie. He should have shot most of it at the traditional 24 FPS but used 48 or 72 for fast motion shots. Hopefully his blunder won't poison high FPS forever in the minds of filmgoers.

> Higher frame rates would help a lot with the blur. I think Peter Jackson made a mistake shooting The Hobbit at 48 FPS for the entire movie. He should have shot most of it at the traditional 24 FPS but used 48 or 72 for fast motion shots. Hopefully his blunder won't poison high FPS forever in the minds of filmgoers.

You can't shoot parts of a film at 24FPS, and parts at 48FPS - the 48FPS parts would be transformed down to 24FPS and would appear to be in "slow motion".

Jackson, for what it's worth, is sticking to his guns re: 48FPS and believes that part of the dislike is because it's "change".

Re: Video Stabilization on YouTube

#49

Post by modeless »

elithrar wrote: post #48
modeless wrote: post #32

Earlier quoted context omitted.

Higher frame rates would help a lot with the blur. I think Peter Jackson made a mistake shooting The Hobbit at 48 FPS for the entire movie. He should have shot most of it at the traditional 24 FPS but used 48 or 72 for fast motion shots. Hopefully his blunder won't poison high FPS forever in the minds of filmgoers.

> Higher frame rates would help a lot with the blur. I think Peter Jackson made a mistake shooting The Hobbit at 48 FPS for the entire movie. He should have shot most of it at the traditional 24 FPS but used 48 or 72 for fast motion shots. Hopefully his blunder won't poison high FPS forever in the minds of filmgoers. You can't shoot parts of a film at 24FPS, and parts at 48FPS - the 48FPS parts would be transformed d…

You absolutely can shoot parts of a film in 24 FPS and parts at 48 FPS. Instead of transforming the 48 FPS parts to 24 FPS, you do the other way around and transform the 24 FPS parts to 48 FPS, not by doubling the speed but by repeating each frame twice. In fact, film projectors have always displayed movies at 48 FPS with frame doubling to reduce the appearance of flicker: http://en.wikipedia.org/wiki/Frame_rate#Background

Re: Video Stabilization on YouTube

#50

Post by nitrogen »

modeless wrote: post #33
CoffeeAndCoffee wrote: post #30

Earlier quoted context omitted.

It also looks like the stabilized video is magnified a little bit. Regardless, these tools are really neat and they are still in their infancy. It's good to see that GooTube still believes in user-generated content.

Cropping and magnifying the video is required for all methods of video stabilization, because otherwise you would see the edge of the original frame move in and out of view.

Are there any stabilization algorithms that use other frames in the past and future to fill in the gaps with (possibly anachronistic) data?
Post reply on HN