Live data from Hacker News

NeRF: Representing scenes as neural radiance fields for view synthesis

matthewtancik.com

31–40 of 43 posts

Re: NeRF: Representing scenes as neural radiance fields for view synthesis

#31
post #25

This blows my mind. This is probably a naive thought; This technique looks like it could be combined with robotics to help it navigate through its environment. I'd also like to see what it does when you give it multiple views of scenes in a video game. Some from the direct pictures and some from pictures of the monitor.

It takes 12 hours on a high end GPU to make one frame.

Re: NeRF: Representing scenes as neural radiance fields for view synthesis

#32

I would like to see “neural enhances” an already rendered 3D scene with the changes which would make it more realistic, given depth map and other information to the neural network

How would it be made more realistic?

Re: NeRF: Representing scenes as neural radiance fields for view synthesis

#33
post #11
post #9

Earlier quoted context omitted.

Huh, what? It needs almost a million views, and takes 1-2 days to train on a GPU. I’m not sure where the “5 minutes” number comes from. EDIT: I was referring to the last paragraph of section 5.3 (Implementation details), but maybe I’m misunderstanding how they use rays / sampled coordinates. Very impressive visual quality. But it seems like they need a LOT of data and computation for each scene. So, its still plausib…

> It needs almost a million views Not sure what you mean by "views". The comparisons in the paper use at most 100 input images per scene.

A pixel is one view for their model if I understand correctly, so one hundred 100x100 images would be a million views.

Re: NeRF: Representing scenes as neural radiance fields for view synthesis

#34
post #25

This blows my mind. This is probably a naive thought; This technique looks like it could be combined with robotics to help it navigate through its environment. I'd also like to see what it does when you give it multiple views of scenes in a video game. Some from the direct pictures and some from pictures of the monitor.

They've only showed it working with static content - they'll need to do it with video (multiple synchronised cameras) and in real time for ant robotics application.

Re: NeRF: Representing scenes as neural radiance fields for view synthesis

#36
post #31
post #25

This blows my mind. This is probably a naive thought; This technique looks like it could be combined with robotics to help it navigate through its environment. I'd also like to see what it does when you give it multiple views of scenes in a video game. Some from the direct pictures and some from pictures of the monitor.

It takes 12 hours on a high end GPU to make one frame.

No, as appendix A of the paper states, each frame takes about 30 seconds to render.

Re: NeRF: Representing scenes as neural radiance fields for view synthesis

#38
post #31

Earlier quoted context omitted.

It takes 12 hours on a high end GPU to make one frame.

No, as appendix A of the paper states, each frame takes about 30 seconds to render.

No, the high dimensional field takes 12 hours and the time to render the field to an image is not going to matter for robotics where computer vision needs to be done in real time.

Re: NeRF: Representing scenes as neural radiance fields for view synthesis

#40
post #34
post #25

This blows my mind. This is probably a naive thought; This technique looks like it could be combined with robotics to help it navigate through its environment. I'd also like to see what it does when you give it multiple views of scenes in a video game. Some from the direct pictures and some from pictures of the monitor.

They've only showed it working with static content - they'll need to do it with video (multiple synchronised cameras) and in real time for ant robotics application.

It'd be interesting to see what happened if they encoded an additional time parameter on each 'view' (input image pixel). Surely someone is already trying to extend this technique that way.
Post reply on HN