Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NeRF: Representing scenes as neural radiance fields for view synthesis

matthewtancik.com

Post reply on HN
Print view View original

21–30 of 43 posts

Re: NeRF: Representing scenes as neural radiance fields for view synthesis

#21

Post by teraflop »

theresistor wrote: post #20
ssivark wrote: post #18

Does anyone know how they do the “virtual object insertion” demonstrated in the paper summary video? Can that be somehow done on the network itself, or is that a diagnostic for scene accuracy by performing SFM on network output?

I'm pretty sure they're rendering a depth channel and compositing it in.

You could do that, but I think it's simpler to just introduce additional objects during the raytracing process that generates the images. That would produce accurate results even with semitransparent objects, unlike compositing with an depth buffer.

Re: NeRF: Representing scenes as neural radiance fields for view synthesis

#22

Post by uoaei »

This is absolutely stunning.

As they say in ML, representation first -- and this is one of the most natural and elegant ways to represent 3D scenes and subjective viewpoints. Great that it works into a rendering environment such that it's E2E differentiable.

This is the first leap toward true high-quality real-time ML-based rendering. I'm blown away.

Re: NeRF: Representing scenes as neural radiance fields for view synthesis

#23

Post by lifeisstillgood »

Well that took some effort just to work out what they actually did. How they actually did it I have no idea. Impressive however - a sort of fill in the blanks for the bits that are missing. If our brains dont do this one would be surprised.

And we are all supposed to become AI developers this decade?!

Come back Visual Basic all is forgiven :-)

Re: NeRF: Representing scenes as neural radiance fields for view synthesis

#25

Post by raidicy »

This blows my mind. This is probably a naive thought; This technique looks like it could be combined with robotics to help it navigate through its environment.

I'd also like to see what it does when you give it multiple views of scenes in a video game. Some from the direct pictures and some from pictures of the monitor.

Re: NeRF: Representing scenes as neural radiance fields for view synthesis

#27

Post by airstrike »

mooneater wrote: post #6
blackhaz wrote: post #2

Could someone ELI5, please?

If you give it a bunch of photos of a scene from different angles, this machine learning method lets you see angles that did not exist in the original set. Better results than other methods so far.

So can we take it to the next level and give it a bunch of ML-generated photos of a scene that doesn't exist (from model B) and let this model A create the 3D view?

Take it one more step further and make model B create photos from some text description similar to the one described in https://news.ycombinator.com/item?id=22640407 (although that one does 3D designs using voxels)

Re: NeRF: Representing scenes as neural radiance fields for view synthesis

#28

Post by iandanforth »

raidicy wrote: post #25

This blows my mind. This is probably a naive thought; This technique looks like it could be combined with robotics to help it navigate through its environment. I'd also like to see what it does when you give it multiple views of scenes in a video game. Some from the direct pictures and some from pictures of the monitor.

Currently view coordinates relative to the volume are required so you first have to solve the SLAM problem before you can optimize a network representation of a given volume.
Post reply on HN