Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia Research Turns 2D Photos into 3D Scenes

blogs.nvidia.com

Post reply on HN
Print view View original

171–180 of 239 posts

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#171

Post by pizza »

nimbleal wrote: post #151
ineedasername wrote: post #62

Earlier quoted context omitted.

"mourn and move on" is a somewhat Western concept of dealing with death. Plenty of cultures around the world have developed different practices, up to religious forms of ancestor worship.

These practices would typically be associated with the idea that the dead have a genuine existence beyond simply “existing in my memory of them” and would be restricted in the forms they take by surrounding ritual, so I’m not sure there’s a direct point of comparison, though?

People often fear losing their memory of a loved one. A memento could help someone heal

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#172

Post by heavyset_go »

baron816 wrote: post #31

It would be really great to recreate loved ones after they have past in some sort of digital space. As I’ve gotten older, and my parents get older as well, I’ve been thinking more about what my life will be like in old age (and beyond too). I’ve also been thinking what I would want “heaven” to be. Eternal life doesn’t appeal to me much. Imagine living a quadrillion years. Even as a god, that would be miserable. That…

I think it would be a special kind of hell to have your resurrected loved ones sell you ads in virtual reality.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#173

Post by preommr »

Zenst wrote: post #155

My first thoughts seeing this is darn, Facebook will with there metaverse, be drinking this up for content. So much so that my thoughts of, would I be shocked if Facebook/Meta made a play to buy Nvidia! Certainly wouldn't shock me as much now as it would before this given how they are banking upon the metaverse/VR being there new growth divergance, what with the leveling of with current services user base after well…

Dude, this isn't 2015 anymore - Nvidia has a higher market cap than Meta.

The days of Meta being a giant that can continously buyout other companies to keep stayling alive are coming to an end.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#174

Post by hansworst »

SV_BubbleTime wrote: post #96
jowday wrote: post #16

Earlier quoted context omitted.

This sort of stuff (generating 3D assets from photographs of real objects) has been common for quite a while via photogrammetry. NeRFs are interesting because (in some cases) they can create renders that look higher quality with fewer photos, and they hint at the potential of future learned rendering models.

I did it for a project… it was SLOW! Gave decent rules for a large area, but poor results for small details. The real trick is textures. When the photo is laid over and wrapped on the model, it looks great. But once you remove that, the raw mesh underneath is not as impressive. I would really like to see the examples they had here without the texture laid on top.

There is no mesh here. Nerfs are 5d (colours are computed based on a 3D position vector + a view direction vector) fields that are rendered volumetrically. So the “texture” is an integral part of the neural representation of the scene, not just an image applied to a mesh.

The cool part is that this also allows for capturing transparency, and any effects caused by lighting (including complex specular reflections) are embedded into the representation.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#175

Post by fxtentacle »

sorenjan wrote: post #44

I don't really understand why NeRFs would be particularly useful in more than a few niche cases, perhaps because I don't fully understand what they really are. My impression is that you take a bunch of photos in various places and directions, then you use those as samples of a 3D function that describes the full scene, and optimize a neural network to minimize the difference between the true light field and what's de…

The pros in the vfx industry still all use reconstructed geometry. And yes, animating or cropping a Nerf is painful.

In my opinion, Nerf is more about showing progress in making AI memorize 3D scenes and the hope is that this will lead to actual understanding sometime in the future.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#176

Post by andybak »

hwers wrote: post #106
daenz wrote: post #2

>The model requires just seconds to train on a few dozen still photos — plus data on the camera angles they were taken from — and can then render the resulting 3D scene within tens of milliseconds. Generating the novel viewpoints is almost fast enough for VR, assuming you're tethered to a desktop computer with whatever GPUs they're using (probably the best setup possible). The holy grail (from my estimation) is getti…

You can just make a mesh once you have the NeRF, which is plenty fast for VR. (What's important isn't new perspective generation but scene training. New perspective generation doesn't have to be real time just within a reasonable time to preprocess and then make a mesh.)

The meshes I've seen from nerfs are pretty horrible and you lose many of the things that make nerfs interesting in the first place.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#177

Post by scoofy »

amelius wrote: post #143
scoofy wrote: post #121

Earlier quoted context omitted.

That's nonsense. There are infinite possible jokes.

This is not true. Jokes that take more than (say) an hour to tell are boring and thus not really jokes.

Literally there are about the same number of atoms in the universe as the number of shuffles in a 52 card deck (52!). The number of topics that can be played with in a joke is beyond even this limited number.

Remember, jokes about tinder/grinder dates didn’t even exist a decade ago.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#179

Post by Synaesthesia »

dharma1 wrote: post #178

In terms of practical use - is there a pipeline to use the NeRF 3D scenes in Unreal Engine? How many photos do you need on average vs photogrammetry? 50% less?

This is not a textured polygon asset, it's a neural field, so it's like stored the directional data in a neural network, I think using spherical harmonics.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#180

Post by matsemann »

idworks1 wrote: post #75
Animats wrote: post #63

What's new about this? That it's faster? People have been reconstructing 3D images from multiple photos for over a decade. The experimental work today is constructing a 3D image from a single photo, using a neural net to fill in a reasonable model of the stuff you can't see.

Five years ago, I've used common software to do this. I had to take hundreds of pictures of a scene, getting as many angles and details possible. Then when you pass it to the computer. Stitching it all together took well beyond 24 hours. Now that I had a 3d model of the scene, I had to spend countless hours cleaning it up to make sure it was useable. Maybe in the last 5 years, things have improved. But this demo used…

If I understand it correctly it didn't make a 3d model, though. So you can't extract and reuse the result. Only move around in it and it creates an image for that viewpoint. But no meshes or textures.
Post reply on HN