41–50 of 152 posts
Im not sure why this demo runs so horribly in Firefox but not other browsers..anyone else having this?
What I haven't seen anything of is feature and object detection, blocking and extraction.
Hopefully a more efficient and streamable codec necessitates the sort of structure that lends itself more easily to analysis.
Wow. Some questions: Take for instance the fulllivingroom demo. (I prefer fps mode.) 1) How many images are input? 2) How long does it take to compute these models? 3) How long does it take to prepare these models for this browser, with all levels, etc? 4) Have you tried this in VR yet?
1) Around 100-150 if memory serves. This scene is part of the mip-NeRF 360 benchmark, which you can download from the corresponding project website: https://jonbarron.info/mipnerf360/
2) Between 12 and 48 hours, depending on the scene. We train on 8x V100s or 16x A100s.
3) The time for preparing assets is included in 2). I don't have a breakdown for you, but it's something like 50/50.
4) Nope! A keen hacker might be able to do this themselves by editing the JavaScript code. Open your browser's DevTools and have a look -- the code is all there!
When might we see this in consumer VR? I'm surprised we don't already but I was suspecting it was a computation constraint. Does this relieve the computation constraint enough to run on Quest 2/3? Is there something else that would prevent binocular use?
What a variety of groups! How did this come about?
How long until you can stitch Street View into a seamless streaming NeRF of every street in the world? I hope that's the goal you're working towards!
Great work!! Question for the authors, are there opportunities, where they exist, to not use optimization or tuning methods for reconstructing a model of a scene? We are refining efficient ways of rendering a view of a scene from these models but the scenes remain static. The scenes also take a while to reconstruct too. Can we still achieve the great look and details of RF and GS without paying for an expensive recon…
If you know a way, let me know! Every system I'm aware of involves optimization in one way or another, from COLMAP to 3D Gaussian Splatting to Instant NGP and more. Optimization is a powerful workhorse that gives us a far wider range of models than a direct solver ever could. > Can we still achieve the great look and details of RF and GS without paying for an expensive reconstruction per instance of the scene?
In the future I hope so. We don't have a convincing way to generate 3D scenes yet, but given the progress in 2D, I think it's only a matter of time.
> Are there ways of greedily reconstructing a scene with traditional CG methods into these new representations now that they are fast to render?
Not that I'm aware of! If there were, I think these works should be on the front page instead of SMERF.
Is there a relatively easy way to apply these kinds of techniques (either NeRFs or gaussian splats) to larger environments even if it's lower precision? Like say small towns/a few blocks worth of env.
Just ran this on my phone through a browser, this is very impressive
My grandpa died 2 years ago and in hindsight I took pictures for using them as in your demo.
Awesome thanks:)