Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Localrf – Nerf from casual shaky videos

localrf.github.io

Post reply on HN
Print view View original

11–20 of 31 posts

Re: Localrf – Nerf from casual shaky videos

#11

Post by potatoman22 »

BugsJustFindMe wrote: post #4

Is this better than what was previously achievable using classical structure from motion? It seems worse because of extreme detail loss? And if they're just plotting a smoothed but similar path through the scenery, didn't Microsoft do that in 2014 with Hyperlapse?

This algorithm constructs a 3D environment from the video data - they're just showcasing it with stabalization. Classical methods require better cameras and more meta. Deep learning is an opportunity for more robust methods for the same end, but also do things like estimating lighting and capturing large scenes.

Re: Localrf – Nerf from casual shaky videos

#15

Post by TaylorAlexander »

BugsJustFindMe wrote: post #4

Is this better than what was previously achievable using classical structure from motion? It seems worse because of extreme detail loss? And if they're just plotting a smoothed but similar path through the scenery, didn't Microsoft do that in 2014 with Hyperlapse?

Structure from motion does not produce good visual fidelity on plants. I’m designing a farming robot and I want remote farmers to be able to view a 3D image of the plants to check for issues, so fidelity is very important. I’ve done a lot of experiments with photogrammetry and NERF, while still presenting a lot of technical challenges, seems far superior for this.

I get the sense that they are mostly using the smoothed views as an example of good results on long scenes. Ultimately the point of nerf is novel/arbitrary view synthesis, which you’re not going to get with Hyperlapse.

And NERF of long tracks is exactly what we need to capture a long row of plants at the farm.

Re: Localrf – Nerf from casual shaky videos

#16

Post by TaylorAlexander »

philipkglass wrote: post #5

My grandmother's farm had to be sold in 2012 after she died. Since my family moved around when I was a kid, but always visited there for holidays, it felt more like home than any other place I lived in. I have extensive videos I recorded in 2006. It'd be wonderful to walk through there again using reconstructions from material I already have. Or maybe not. There's a reason I haven't watched those videos in years. Who…

I think it depends on where you are in life. If your life is good and you feel safe and happy, then looking back can be a nice way to remember your childhood, and possibly rediscover things about yourself you had forgotten. If your life is difficult and you’re not feeling great, looking back at better times can be a painful reminder of things not going the way you’d like. But our lives ebb and flow, and there may be a time where you feel like looking back.

Re: Localrf – Nerf from casual shaky videos

#17

Post by regularfry »

Gordonjcp wrote: post #14

When I've got my glasses on, the "Forest" sequences look *wildly* 3D to me. Without my glasses it looks less so but that might be a function of not really being able to focus on stuff closer than about ten metres ;-)

I think what you're getting a demo of there is the bit of our depth perception apparatus that doesn't need stereo pairs to work. Out past a certain distance our brains reconstruct geometry from visual depth cues, and I suspect that the smoothed camera path means that, in contrast to the shaky original video, your eyes are seeing something your inner ears aren't contradicting too much.

Re: Localrf – Nerf from casual shaky videos

#18

Post by v7n »

Let's say you want to slightly alter the spline the camera of the 3D scene moves along, but the positions/angles would change in such a way there is missing data in the new rendering. How feasible would it be to use current inpainting technologies to fill in the gaps straight in the scene? Would it be better to try to inpaint the rendered frames instead?
Post reply on HN