Live data from Hacker News

First-Person Hyper-lapse Videos

research.microsoft.com

131–140 of 184 posts

Re: First-Person Hyper-lapse Videos

#131

The videos don't load for me (due to mixed content, I believe), so here they are: https://www.youtube.com/watch?v=SOpwHaQnRSY https://www.youtube.com/watch?v=sA4Za3Hv6ng The hyperlapse of the climbing video looks like an FPS game from a decade ago with texture refreshing as you get closer.

Hmm... Did I not embed these correctly? The project page as well as the embed links are plain http:// this is what it should be, no?

Re: First-Person Hyper-lapse Videos

#132
post #64

Earlier quoted context omitted.

Take a look at an older paper 'Content-Preserving Warps for 3D Video Stabilization'. That seems to do an ok job http://createdigitalmotion.com/2009/06/magical-3d-warping-te...

EDIT: After clicking through to their website, I found this updated technique which shares one principal author that seems to work even better: http://people.cs.nctu.edu.tw/~yushuen/VideoStabilization/ Wow. Actually, if they that add that technique to the mix it might solve the deformed "pop"-effect you see in some videos, like the deformed building you see around 16 seconds into this videos: http://research.microsof…

Cool, I'd not seen the updated work. I wonder how much can be done in realtime. I have no idea what the compute split is between between the different processes.

With sensors (gyros etc) the camera path would be trivial, instead of recovering that from the video. Rendering the results would be possible on a mobile GPU. Just leaving the frame conversion to a point cloud in terms of compute and memory.

Maybe some scheme where you down sample the input frames to create the deformation mesh, then apply that to the full size frame would be the way to go

Re: First-Person Hyper-lapse Videos

#133
post #131

The videos don't load for me (due to mixed content, I believe), so here they are: https://www.youtube.com/watch?v=SOpwHaQnRSY https://www.youtube.com/watch?v=sA4Za3Hv6ng The hyperlapse of the climbing video looks like an FPS game from a decade ago with texture refreshing as you get closer.

Hmm... Did I not embed these correctly? The project page as well as the embed links are plain http:// this is what it should be, no?

People using the HTTPS Everywhere extension will have the site URL as https, but not the embedded videos.

Re: First-Person Hyper-lapse Videos

#134
post #132

Earlier quoted context omitted.

EDIT: After clicking through to their website, I found this updated technique which shares one principal author that seems to work even better: http://people.cs.nctu.edu.tw/~yushuen/VideoStabilization/ Wow. Actually, if they that add that technique to the mix it might solve the deformed "pop"-effect you see in some videos, like the deformed building you see around 16 seconds into this videos: http://research.microsof…

Cool, I'd not seen the updated work. I wonder how much can be done in realtime. I have no idea what the compute split is between between the different processes. With sensors (gyros etc) the camera path would be trivial, instead of recovering that from the video. Rendering the results would be possible on a mobile GPU. Just leaving the frame conversion to a point cloud in terms of compute and memory. Maybe some schem…

> With sensors (gyros etc) the camera path would be trivial, instead of recovering that from the video.

Well... not quite trivial. They're calibrated differently per model, and it's actually quite tricky to reconstruct the path based on accelerometers and gyroscope alone. There's also the likely issue of synchronising the data from these sensors with the video input. If you solve that second issue however, it could in theory at least help with recovering the path from the video, creating better predictions where the point cloud has moved to for example.

Re: First-Person Hyper-lapse Videos

#135
post #32
post #17

Bloody amazing! Fantastic work! Release it. Release it soon. This is something that everyone would want. I see they have listed a Windows app coming. Is that Windows desktop app?

If I were them I would offer this as a web service: upload your video and have it converted for $1.

Someone above mentioned the processing took weeks. So.. I'm not sure that would work.

Re: First-Person Hyper-lapse Videos

#136
post #32
post #17

Bloody amazing! Fantastic work! Release it. Release it soon. This is something that everyone would want. I see they have listed a Windows app coming. Is that Windows desktop app?

If I were them I would offer this as a web service: upload your video and have it converted for $1.

[deleted]

Re: First-Person Hyper-lapse Videos

#137
As others have commented, the videos look great, and much closer to how people remember journeys. However, there appear to be some image persistence problems (many street poles simply dissolve as they get closer to the camera).

I'm curious to see what happens if they insert more action-packed footage. An MTB course with trees, switchbacks, and jumps would be an interesting stress test of this technique.

Re: First-Person Hyper-lapse Videos

#139
This is great. I have weeks of footage from a camera that I wear around and would love to use that video to make a hyperlapse. I would also be interested in seeing how well this does with photos taken every few seconds as opposed to video. Although, after reading the paper, it looks like there would be a lot of optimization that would need to happen to make it more efficient. (Their original implementation took a few hours on a cluster.) Luckily, as they stated in the technical video, they haven't tried to do anything more than a proof of concept; so there is plenty of room to optimize. I'd be interested to see how well a single-machine OpenCL or CUDA implementation does compared to the CPU clusters they were using in the paper.

Re: First-Person Hyper-lapse Videos

#140
post #132

Earlier quoted context omitted.

Cool, I'd not seen the updated work. I wonder how much can be done in realtime. I have no idea what the compute split is between between the different processes. With sensors (gyros etc) the camera path would be trivial, instead of recovering that from the video. Rendering the results would be possible on a mobile GPU. Just leaving the frame conversion to a point cloud in terms of compute and memory. Maybe some schem…

> With sensors (gyros etc) the camera path would be trivial, instead of recovering that from the video. Well... not quite trivial. They're calibrated differently per model, and it's actually quite tricky to reconstruct the path based on accelerometers and gyroscope alone. There's also the likely issue of synchronising the data from these sensors with the video input. If you solve that second issue however, it could i…

I was being a little cheeky with that. I know position and orientation data can be quite noisy and getting clean good quality data is not simple
Post reply on HN