Are there examples of this being used on large outdoor spaces?
Yes, Waymo did the whole San Francisco block: https://waymo.com/research/block-nerf/
Woah, this video is way more interesting than the Nvidia polaroid teaser in the original link.
Nvidia Research Turns 2D Photos into 3D Scenes
21–30 of 239 posts
Re: Nvidia Research Turns 2D Photos into 3D Scenes
#22My prediction/hope is that NeRFs will totally revolutionize how the film/TV industry. I can imagine: - Shooting a movie from a few cameras, creating a movie version of a NeRF using those angles, and then dynamically adding in other shots in post - Using lighting and depth information embedded in NeRFs to assist in lighting/integrating CG elements - Using NeRFs to generate virtual sets on LED walls (like those on The…
Perhaps even making non-gimmicky live action 3d films.
Having 3d renders of the entire film without needing green screens and a bunch of balls seems like it would have to make some of the post processing work easier. You can add or remove elements. Adjust the camera angles. More effectively de-age actors. Heck, even create scenes whole cloth if an actor unexpectedly dies (since you still have their model).
Seems like you could also save some time having fewer takes. What you can fix in post would be dramatically expanded.
Best part for film makers, they are often using multiple cameras anyways. So this doesn't seem like it'd be too much of a stretch.
Re: Nvidia Research Turns 2D Photos into 3D Scenes
#23My prediction/hope is that NeRFs will totally revolutionize how the film/TV industry. I can imagine: - Shooting a movie from a few cameras, creating a movie version of a NeRF using those angles, and then dynamically adding in other shots in post - Using lighting and depth information embedded in NeRFs to assist in lighting/integrating CG elements - Using NeRFs to generate virtual sets on LED walls (like those on The…
Computer games, VR and AR could also be pretty amazing uses for this technique too.
RIP photo-realistic modelers
Re: Nvidia Research Turns 2D Photos into 3D Scenes
#24My prediction/hope is that NeRFs will totally revolutionize how the film/TV industry. I can imagine: - Shooting a movie from a few cameras, creating a movie version of a NeRF using those angles, and then dynamically adding in other shots in post - Using lighting and depth information embedded in NeRFs to assist in lighting/integrating CG elements - Using NeRFs to generate virtual sets on LED walls (like those on The…
My, maybe too extreme, future fantasy version of this is turning existing movies into 3d movies you could watch in VR.
Re: Nvidia Research Turns 2D Photos into 3D Scenes
#25Earlier quoted context omitted.
Yes, Waymo did the whole San Francisco block: https://waymo.com/research/block-nerf/
Woah, this video is way more interesting than the Nvidia polaroid teaser in the original link.
Still, NVIDIA's achievement (and Thomas Müller in particular) is amazing. Thomas and his collaborators achieved an almost 1000x performance improvement, by a combination of algorithmical and implementation tricks.
I highly recommend trying this at home:
https://nvlabs.github.io/instant-ngp/
https://github.com/NVlabs/instant-ngp
Very straightforward and gives better insight into what NeRF is than any shiny marketing demo.
Re: Nvidia Research Turns 2D Photos into 3D Scenes
#26If you have a graphics card which is unobtainable.
Re: Nvidia Research Turns 2D Photos into 3D Scenes
#27Tangent I wonder what happens to most people when they see innovation such as this. Over the years I have seen numerous mind-blowing AI achievement, which essentially feel like miracles. Yet literally after an hour I forget what I even saw. I don't find these innovations to have a lasting impression on me or on the internet except for the times when these solutions are released to the public for tinkering and they en…
I have the impression, that now some of them seem to really end up in some practical applications. Funnily enough someone just today showed me a feature of his phone where you can select some undesired objects in youe photo and it would just replace them with a fitting background indistinguishable from the original photo.
Re: Nvidia Research Turns 2D Photos into 3D Scenes
#28I know that taste in comedy is seasonal (yes, there were a people in a time that thought vaudeville was the cat's pajamas), but has anyone ever greeted a pun with anything other than a pained sigh?
Puns aren't to make people laugh, the pained sigh is the point. It's schadenfreude for the person making the pun.
Re: Nvidia Research Turns 2D Photos into 3D Scenes
#29Earlier quoted context omitted.
Computer games, VR and AR could also be pretty amazing uses for this technique too.
RIP photo-realistic modelers
hmmm well I still think they will be in demand for the same reason software developers will be not automated away. NeRF is really mind boggling good but there are still artifacts, and something that modelers have a good eye for.
Having said that, it might be the end for any junior type of roles. Same reason that github copilot really takes a bite of the need to have a junior developer.
I'm very curious what will happen because it will become a sort of trend across other industries apart from legal or medical professions (peace of mind from human-in-the-loop).
Re: Nvidia Research Turns 2D Photos into 3D Scenes
#30My prediction/hope is that NeRFs will totally revolutionize how the film/TV industry. I can imagine: - Shooting a movie from a few cameras, creating a movie version of a NeRF using those angles, and then dynamically adding in other shots in post - Using lighting and depth information embedded in NeRFs to assist in lighting/integrating CG elements - Using NeRFs to generate virtual sets on LED walls (like those on The…
> - Using lighting and depth information embedded in NeRFs to assist in lighting/integrating CG elements
> - Using NeRFs to generate virtual sets on LED walls
Sounds like a powerful set of tools to defeat a number of image manipulation detection tricks, with limited effort once the process is set up as routine. State actor level information warfare will soon be a class of its own. Not just in terms of getting harder to detect, but more importantly in terms of becoming able to produce "quality" in high volume.