Live data from Hacker News

First-Person Hyper-lapse Videos

research.microsoft.com

51–60 of 184 posts

Re: First-Person Hyper-lapse Videos

#51

It appears as if the mountains are loading in the background (like in a video game) as you get closer to them. Awesome idea/execution!

I guess that it's not a proximity problem. For example, in the first video at 3:08 a gray mountain with snow appears on the upper right corner, and replaces a piece of sky. I think that a big rock was occluding the vision of the mountain in this frame, and the algorithm has to choose a texture from another frame to fill the void, and it made a mistake.

Re: First-Person Hyper-lapse Videos

#52

The result is quite simply breathtaking. It looks like something shot for a movie using a stabilised dollycam, the fact they were able to achieve the same thing using nothing but a GoPro, their software, and likely a week of post-processing on a high end desktop PC is simply amazing. I hope we see this technology actually become readily available. There might still be work to be done, but in general if they can repro…

It's striking but it's far more believable when you realize that they need to play at a much faster speed than the source, so they have tons of extra samples from which to extract information. They basically use all that data in the extra frames (that would otherwise simply get tossed away in a regular time-lapse) to construct a 3D scene. This wouldn't look nearly as good if they had to play it at normal speed.

Re: First-Person Hyper-lapse Videos

#54
post #45

Video stabilization + more FPS / slower rate than the "every 10 frames timelapse" + feel good inspirational music = this I would guess that I could upload a shaky video to youtube to get it smoothed out, download it, and speed it up with similar to their rate and get similar results. The timelapse that they show that is so much worse uses way less frames of the raw footage (every 10th frame?) and goes way faster than…

>I would guess that I could upload a shaky video to youtube to get it smoothed out, download it, and speed it up with similar to their rate and get similar results.

No you certainly wouldn't. Watch the technical video at the bottom of the page. It will explain why this is not trivial to do and why standard stabilisation technologies aren't useful to smooth out time lapses.

Re: First-Person Hyper-lapse Videos

#56
post #45

Video stabilization + more FPS / slower rate than the "every 10 frames timelapse" + feel good inspirational music = this I would guess that I could upload a shaky video to youtube to get it smoothed out, download it, and speed it up with similar to their rate and get similar results. The timelapse that they show that is so much worse uses way less frames of the raw footage (every 10th frame?) and goes way faster than…

If you'd looked at the site, the authors actually do the exact comparison you suggest: http://research.microsoft.com/en-us/um/redmond/projects/hype... It's better than a raw timelapse, but not (IMHO) nearly as good as their results.

From the paper intro:

    Video stabilization algorithms could conceivably help create smoother
    hyper-lapse videos. Although there has been significant recent
    progress in video stabilization techniques (see Section 2),they do not
    perform well on casually captured hyper-lapse videos. The dramatically
    increased camera shake makes it difficult to track the motion between
    successive frames. Also, since all methods operate on a
    single-frame-in-single-frame-out basis, they would require dramatic
    amounts of cropping. Applying the video stabilization before
    decimating frames also does not work because the methods use
    relatively short time windows, so the amount of smoothing is
    insufficient to achieve smooth hyper-lapse results.
And later on (section 7.1):

    As mentioned in our introduction, we also experimented with
    traditional video stabilization techniques, applying the stabilization
    both before and after the naive time-lapse frame decimation step. We
    tried several available algorithms, including the Warp Stabilizer in
    Adobe After Effects, Deshaker 1, and the Bundled Camera Paths method
    [Liu et al. 2013]. We found that they all produced very similar
    looking results and that neither variant (stabilizing before or after
    decimation) worked well, as demonstrated in our supplementary
    material. We also tried a more sophisticated temporal coarse-to-fine
    stabilization technique that stabilized the original video, then
    subsampled the frames in time by a small amount, and then repeated
    this process until the desired video length was reached. While this
    approach worked better than the previous two approaches (see the
    video), it still did not produce as smooth a path as the new technique
    developed in this paper, and significant distortion and wobble
    artifacts accumulated due to the repeated application of
    stabilization.

Re: First-Person Hyper-lapse Videos

#57
If the MSR researchers are here -- I'm curious what does it look like when bordering hyperlapse with regular input? i.e., if there were a video consisting of input frames at the beginning and the end, with a stretch of hyperlapse in the middle, what does the transition look like? Does it need much smoothing?

Also you probably saw this over the past week: http://jtsingh.com/index.php?route=information/information&i... (disregarding the politics of that) Whatever he's doing (I assume a lot of manual work) it has a very similar effect and it has these beautiful transitions between speeds.

Amazing work and the videos are stunning.

Re: First-Person Hyper-lapse Videos

#59
post #47
post #44

Nice results, but missing forest for the trees. One of the by-products of this algorithm is fully textured 3d model representing filmed environment. Offering that as pure data dump, or even a manual process allowing user to control camera would be as valuable as fully automatic one-off timelapse no one ever watches (except maybe your granny). What sounds better - a video tour of a house, or a 3D model of a house you…

Generating a 3D model of an environment from the output of a moving camera has been done. There is obviously a lot of improvement to be done in that department, and those projects are neat, but I think it's appropriate for this project to focus on what it adds to the scene, which is camera path smoothing.

Indeed it has, photosynth - I blogged about it before, http://alicious.com/making-3d-models-from-photos/, http://photosynth.net/preview/view/6de2eb38-4e7e-424b-94a3-9... - which came from Microsoft Research. It looks like this is "just" the video version of what they were doing before with photosynth walkthroughs. Very impressive.

Re: First-Person Hyper-lapse Videos

#60

The result is quite simply breathtaking. It looks like something shot for a movie using a stabilised dollycam, the fact they were able to achieve the same thing using nothing but a GoPro, their software, and likely a week of post-processing on a high end desktop PC is simply amazing. I hope we see this technology actually become readily available. There might still be work to be done, but in general if they can repro…

Yeah, it looks amazing. If the video is taken at faster speed (like 10X), then they can get a smooth realtime result when slowing it down at post processing.
Post reply on HN