Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia Research Turns 2D Photos into 3D Scenes

blogs.nvidia.com

Post reply on HN
Print view View original

191–200 of 239 posts

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#191

Post by Kerrick »

Groxx wrote: post #187
fleischhauf wrote: post #27

Earlier quoted context omitted.

I have the impression, that now some of them seem to really end up in some practical applications. Funnily enough someone just today showed me a feature of his phone where you can select some undesired objects in youe photo and it would just replace them with a fitting background indistinguishable from the original photo.

It's rather entertaining when this happens in the opposite direction automatically too: https://twitter.com/mitchcohen/status/1476351601862483968

The followup on that is... it didn't happen. There was a leaf in the foreground, and the depth of field in the photo was large enough that it was in focus rather than blurring. https://twitter.com/mitchcohen/status/1476951534160257026

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#192

Post by efraim »

Groxx wrote: post #187
fleischhauf wrote: post #27

Earlier quoted context omitted.

I have the impression, that now some of them seem to really end up in some practical applications. Funnily enough someone just today showed me a feature of his phone where you can select some undesired objects in youe photo and it would just replace them with a fitting background indistinguishable from the original photo.

It's rather entertaining when this happens in the opposite direction automatically too: https://twitter.com/mitchcohen/status/1476351601862483968

In a follow up tweet it looks like there was a leaf on a tree in the foreground that obscured her face, not an AI replacement.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#193

Post by amelius »

Groxx wrote: post #187
fleischhauf wrote: post #27

Earlier quoted context omitted.

I have the impression, that now some of them seem to really end up in some practical applications. Funnily enough someone just today showed me a feature of his phone where you can select some undesired objects in youe photo and it would just replace them with a fitting background indistinguishable from the original photo.

It's rather entertaining when this happens in the opposite direction automatically too: https://twitter.com/mitchcohen/status/1476351601862483968

It was already debunked. If you scroll down in that Twitter thread, you read:

> Big news! I sent @sdw the original image. He theorized a leaf from a foreground tree obscured the face. I didn’t think anything was in view, and the closest tree is a Japanese Maple (smaller leaves). But he’s right! Here’s a video I just shot, showing the parallax. Wow!

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#194

Post by shultays »

Is the example the result of just 4 photos? Or more? Are there any other data available, spatial data attached to photos for example?

Why they don't explain the scope of achievement properly?

edit: I don't think it is just 4 https://news.ycombinator.com/item?id=30810885

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#196

Post by neveradmitmyid »

olladecarne wrote: post #46
baron816 wrote: post #31

It would be really great to recreate loved ones after they have past in some sort of digital space. As I’ve gotten older, and my parents get older as well, I’ve been thinking more about what my life will be like in old age (and beyond too). I’ve also been thinking what I would want “heaven” to be. Eternal life doesn’t appeal to me much. Imagine living a quadrillion years. Even as a god, that would be miserable. That…

There's so many things we invent with good intentions but in the end go terribly wrong and I think this is one of those things. I think it's ok to mourn and remember the past, but moving on and accepting reality is important to a healthy life. Let's be real though, the startup that makes this but appeals to our worst instincts make bank. I can't imagine how much more messed up future generations will be as we keep ma…

Let be real: I've worked on the R&D stage of a Chinese research project for a State supported Ancestor Worship software where people's ancestors are recreated in 3D, their "ancestorial home" is made available in pieces and parts the software user must purchase with real currency, and the user is encouraged to discuss their day to day life issues with their observing and consoling animated ancestors. The software is a complete Orwellian Spy while masquerading as all your ancestors listening, offering advice, and demanding gifts that cost real currency. To say the least, I spooked the hell out of that situation.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#197

Post by shultays »

syspec wrote: post #20

Is there a video of this? I'm not sure what's the connection to the top photo/video/matrix-360-effect Was that created from a few photos? I didn't see any additional imagery below --- Update It looks like these are the four source photos: https://blogs.nvidia.com/wp-content/uploads/2022/03/NVIDIA-R... Then it creates this 360 video from them: https://blogs.nvidia.com/wp-content/uploads/2022/03/2141864_...

    four source photos
Is it just 4 or are there more?

I find it hard to believe there is only 4. There are clearly more data in video

https://i2.paste.pics/645fe17e418b2cb1f6179e0b6671a170.png like back side of camera here (it is kinda visible but much poorer compared to video). Or existence of a 2nd white sheet in background. But correct me if it is only 4 and you have a source on that

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#199

Post by shultays »

idworks1 wrote: post #75
Animats wrote: post #63

What's new about this? That it's faster? People have been reconstructing 3D images from multiple photos for over a decade. The experimental work today is constructing a 3D image from a single photo, using a neural net to fill in a reasonable model of the stuff you can't see.

Five years ago, I've used common software to do this. I had to take hundreds of pictures of a scene, getting as many angles and details possible. Then when you pass it to the computer. Stitching it all together took well beyond 24 hours. Now that I had a 3d model of the scene, I had to spend countless hours cleaning it up to make sure it was useable. Maybe in the last 5 years, things have improved. But this demo used…

Did it really use only 4 pictures? Do you have a source? https://news.ycombinator.com/item?id=30810885

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#200

Post by yorwba »

wokwokwok wrote: post #103
p1esk wrote: post #59

Earlier quoted context omitted.

What happens if you want to animate part of a NeRF, or crop it, or change it in any way? Do you have to throw away all trained coefficients and start again from training data? You don’t change NeRF (the model). You change the point of view of an observer.

I mean, this is the parent posts point; the use case for a static photo or a static 3d nerf is pretty limited. With other structured data compositing and animating is relatively trivial. It turns out that people have approached this problem before and you can composite nerf too (1) by sampling different functions over the volume. …but, let’s not pretend. The complaint is entirely valid. You’re taking a high resolutio…

> Working with simple voxel data let’s you do all kinds of normal image manipulation techniques, and it’s not clear how you would do some of those with a nerf.

Any image transformation you can do on voxels you can straightforwardly transfer to nerfs. Voxel data is just a lookup table from discrete positions to material properties like color and density. When you apply a transform, you change the inputs (e.g. multiplying them with a rotation matrix) or the outputs (e.g. changing the color). If you want to do the same thing with a nerf that maps continuous positions and directions to material properties like color and density, just transform the inputs or the outputs.

The major difference is that with voxel data you can easily do output-modifying transformations directly on the stored representation, while for nerfs it might be cheaper to do it on the fly instead of redoing the training procedure to bake the change into the model.

Post reply on HN