Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia Research Turns 2D Photos into 3D Scenes

blogs.nvidia.com

Post reply on HN
Print view View original

71–80 of 239 posts

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#71

Post by 1024core »

rilezg wrote: post #45
baron816 wrote: post #31

It would be really great to recreate loved ones after they have past in some sort of digital space. As I’ve gotten older, and my parents get older as well, I’ve been thinking more about what my life will be like in old age (and beyond too). I’ve also been thinking what I would want “heaven” to be. Eternal life doesn’t appeal to me much. Imagine living a quadrillion years. Even as a god, that would be miserable. That…

This reminds me a lot of Black Mirror season 2 episode 1. Always good to treasure the time we are given.

I miss Black Mirror. Some episodes were hit-n-miss, but many of them made me think. The best part was that each episode was mostly independent of the other.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#72

Post by 1024core »

elil17 wrote: post #8

My prediction/hope is that NeRFs will totally revolutionize how the film/TV industry. I can imagine: - Shooting a movie from a few cameras, creating a movie version of a NeRF using those angles, and then dynamically adding in other shots in post - Using lighting and depth information embedded in NeRFs to assist in lighting/integrating CG elements - Using NeRFs to generate virtual sets on LED walls (like those on The…

I can see this really taking off for football games. You'll be able to look at plays from all angles, zoom in/out, and get to _play_ with the game.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#73

Post by mkaic »

as someone who works in both AI and filmmaking, I remember losing my mind when this paper was first released a few weeks ago. It's absolute insanity what the folks at Nvidia have managed to accomplish in such a short time. The paper itself[0] is quite dense, but I recommend reading it -- they had to pull some fancy tricks to get performance to be as good as it is!

[0]https://nvlabs.github.io/instant-ngp/

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#74

Post by celim307 »

baron816 wrote: post #31

It would be really great to recreate loved ones after they have past in some sort of digital space. As I’ve gotten older, and my parents get older as well, I’ve been thinking more about what my life will be like in old age (and beyond too). I’ve also been thinking what I would want “heaven” to be. Eternal life doesn’t appeal to me much. Imagine living a quadrillion years. Even as a god, that would be miserable. That…

Isn’t that the premise of the Amazon series upload

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#75

Post by idworks1 »

Animats wrote: post #63

What's new about this? That it's faster? People have been reconstructing 3D images from multiple photos for over a decade. The experimental work today is constructing a 3D image from a single photo, using a neural net to fill in a reasonable model of the stuff you can't see.

Five years ago, I've used common software to do this. I had to take hundreds of pictures of a scene, getting as many angles and details possible. Then when you pass it to the computer. Stitching it all together took well beyond 24 hours.

Now that I had a 3d model of the scene, I had to spend countless hours cleaning it up to make sure it was useable. Maybe in the last 5 years, things have improved.

But this demo used 4 pictures. And apparently, it rendered the final image in seconds. That's what's new.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#76

Post by renewiltord »

noduerme wrote: post #35
krasin wrote: post #7

Earlier quoted context omitted.

Yes, Waymo did the whole San Francisco block: https://waymo.com/research/block-nerf/

There's something very uncanny-valley about that video. I can't decide if it's the smoothness of the shading on the textures or if it's the way the parallax perspective on the buildings sometimes is just a tiny bit off. I don't generally get motion sickness from VR but I feel like this would cause it.

Could just be the lighting on the reconstruction. It's not meant to be photorealistic. It's freaking nuts how they can do this.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#77

Post by dylan604 »

jowday wrote: post #55
noduerme wrote: post #35

Earlier quoted context omitted.

There's something very uncanny-valley about that video. I can't decide if it's the smoothness of the shading on the textures or if it's the way the parallax perspective on the buildings sometimes is just a tiny bit off. I don't generally get motion sickness from VR but I feel like this would cause it.

You’ll find this is true of all NeRFs if you spend time playing around with them. If a NeRF is trying to render part of an object that wasn’t observered in the input images, it’s going to look strange, since it’s ultimately just guessing at the appearance. The NVidia example in the link has the benefit of focusing on a single entity that’s centered in all of the input photographs - the effect is much more pronounced…

I know when doing typical 2D video based rotoscoping it is possible to use frames from before/after the current frame to see data that is being blocked in the current frame. It's also common in restoration when removing scrathes/hair in the gate/etc.

To that end, I wonder if exporting a similar bit of video from that same path exported as stills would be enough to generate the 3D version.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#78

Post by [deleted] »

olladecarne wrote: post #46
baron816 wrote: post #31

It would be really great to recreate loved ones after they have past in some sort of digital space. As I’ve gotten older, and my parents get older as well, I’ve been thinking more about what my life will be like in old age (and beyond too). I’ve also been thinking what I would want “heaven” to be. Eternal life doesn’t appeal to me much. Imagine living a quadrillion years. Even as a god, that would be miserable. That…

There's so many things we invent with good intentions but in the end go terribly wrong and I think this is one of those things. I think it's ok to mourn and remember the past, but moving on and accepting reality is important to a healthy life. Let's be real though, the startup that makes this but appeals to our worst instincts make bank. I can't imagine how much more messed up future generations will be as we keep ma…

[deleted]

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#79

Post by jamiek88 »

alanwreath wrote: post #3

I’m probably going to ramp up the number of photos I take in hope that google photos auto applies this tech

I’m gonna do the same particularly dog pics. I’d love to bring my old dogs back to virtual life. Have old spanky (died a decade ago) running with lilybean (current version) in iMovie.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#80

Post by tonguez »

baron816 wrote: post #31

It would be really great to recreate loved ones after they have past in some sort of digital space. As I’ve gotten older, and my parents get older as well, I’ve been thinking more about what my life will be like in old age (and beyond too). I’ve also been thinking what I would want “heaven” to be. Eternal life doesn’t appeal to me much. Imagine living a quadrillion years. Even as a god, that would be miserable. That…

“I’ve also been thinking what I would want “heaven” to be. Eternal life doesn’t appeal to me much.”

please tell me more!

Post reply on HN