21–30 of 31 posts
Can't wait for this stuff to become so good and polished behind a UI, so that I can just point it to a folder of videos and see what comes out.
nice edit: is this process public? how can i test and try this? on kdenlive? edit 2: https://github.com/facebookresearch/localrf got it but when will it be implemented by kdenlive? any dev here?
My grandmother's farm had to be sold in 2012 after she died. Since my family moved around when I was a kid, but always visited there for holidays, it felt more like home than any other place I lived in. I have extensive videos I recorded in 2006. It'd be wonderful to walk through there again using reconstructions from material I already have. Or maybe not. There's a reason I haven't watched those videos in years. Who…
I think it depends on where you are in life. If your life is good and you feel safe and happy, then looking back can be a nice way to remember your childhood, and possibly rediscover things about yourself you had forgotten. If your life is difficult and you’re not feeling great, looking back at better times can be a painful reminder of things not going the way you’d like. But our lives ebb and flow, and there may be…
And their main takeaway seems to be that one should do global bundle adjustment for recovering the camera poses ... which I thought has been common knowledge for years and is what pretty much every SfM tool implements.
My TLDR would be: stuff that works well continues to work well even if you use a neural radiance field instead of a point cloud for representing geometry.
Those results look eerily similar to Microsoft's 2016 Hyperlapse paper&software.
When I've got my glasses on, the "Forest" sequences look *wildly* 3D to me. Without my glasses it looks less so but that might be a function of not really being able to focus on stuff closer than about ten metres ;-)
I think what you're getting a demo of there is the bit of our depth perception apparatus that doesn't need stereo pairs to work. Out past a certain distance our brains reconstruct geometry from visual depth cues, and I suspect that the smoothed camera path means that, in contrast to the shaky original video, your eyes are seeing something your inner ears aren't contradicting too much.
I wonder if maybe their implementation "fills in the blanks" in the 3D space in the same way our brain does, so it looks "properly 3D" because it's what we're expecting it to look like already?
Let's say you want to slightly alter the spline the camera of the 3D scene moves along, but the positions/angles would change in such a way there is missing data in the new rendering. How feasible would it be to use current inpainting technologies to fill in the gaps straight in the scene? Would it be better to try to inpaint the rendered frames instead?
For those like me who didn't know what NeRF means - Neural Radiance Fields. [1] https://www.matthewtancik.com/nerf [2] https://datagen.tech/guides/synthetic-data/neural-radiance-f...