Nvidia Research Turns 2D Photos into 3D Scenes • Hacker News Board

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#81

Post by dylan604 » Fri, Mar 25, 2022, 11:41 PM UTC

alanwreath wrote: post #3

I’m probably going to ramp up the number of photos I take in hope that google photos auto applies this tech

Google has done some expirements from people taking very similarly framed images from different times to create timelapse videos, so that would hint to me they definitely have the content to try.

There was an article here some time back that showed streets in NYC that used this kind of idea of using older photographs to put one in street view of older NYC. So, yeah, I'm guessing it could be done. Might be weird with the different quality of images (modern digital, polaroids, kodachrome, etc).

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#82

Post by jvanderbot » Fri, Mar 25, 2022, 11:41 PM UTC

> Creating a 3D scene with traditional methods takes hours or longer, depending on the complexity and resolution of the visualization.

This just isn't true. I can create a 3D scene from 360-degree photos (even 4) in a minute or so using traditional methods, even open-source toolkits.

It doesn't look as good as this because it doesn't have a neural net smoothing the gaps, but it's not true that it takes hours to build 3D information from 2D images.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#83

Post by wubbert » Fri, Mar 25, 2022, 11:41 PM UTC

IIRC Microsoft had something like this years ago, but the results weren't nearly as smooth or natural looking. I can't remember what it was called, though.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#84

Post by [deleted] » Fri, Mar 25, 2022, 11:41 PM UTC

lowdest wrote: post #47

baron816 wrote: post #31

It would be really great to recreate loved ones after they have past in some sort of digital space. As I’ve gotten older, and my parents get older as well, I’ve been thinking more about what my life will be like in old age (and beyond too). I’ve also been thinking what I would want “heaven” to be. Eternal life doesn’t appeal to me much. Imagine living a quadrillion years. Even as a god, that would be miserable. That…

>Imagine living a quadrillion years. Even as a god, that would be miserable. This seems very subjective, I don't agree at all.

[deleted]

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#85

Post by jamiek88 » Fri, Mar 25, 2022, 11:41 PM UTC

XorNot wrote: post #33

So the part which makes this interesting to me is the speed. My new desire in our video conferencing world these days has been to have my camera on but running a corrected model of myself so I can sustain apparently eye-contact without needing to look directly at the camera.

https://osxdaily.com/2021/05/12/enable-eye-contact-facetime-...

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#86

Post by airstrike » Fri, Mar 25, 2022, 11:43 PM UTC

baron816 wrote: post #31

It would be really great to recreate loved ones after they have past in some sort of digital space. As I’ve gotten older, and my parents get older as well, I’ve been thinking more about what my life will be like in old age (and beyond too). I’ve also been thinking what I would want “heaven” to be. Eternal life doesn’t appeal to me much. Imagine living a quadrillion years. Even as a god, that would be miserable. That…

Hah! I made this suggestion here about a couple years ago. It's been my dream-pet-project-I'll-never-create for a while now, but it seems like more talented and predisposed individuals have also considered it / may be working on it already...

Some interesting replies there

https://news.ycombinator.com/item?id=24073994

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#87

Post by pvarangot » Fri, Mar 25, 2022, 11:43 PM UTC

baron816 wrote: post #31

It would be really great to recreate loved ones after they have past in some sort of digital space. As I’ve gotten older, and my parents get older as well, I’ve been thinking more about what my life will be like in old age (and beyond too). I’ve also been thinking what I would want “heaven” to be. Eternal life doesn’t appeal to me much. Imagine living a quadrillion years. Even as a god, that would be miserable. That…

If that technology existed a lot of people would live in misery knowing that they could be seeing their dog and their parents and speak to them one last time, but only when their time to goes come. Either that or people would be able to use it whenever they wanted or could pay for it and it would become a new highly addictive and harmfully dissociative tech product that I venture to guess would be like 10x more harmful to society than Facebook ever was.

I don't even know if it would have some kind of therapeutic use of whatever. Yeah ir would be a wonderful experience and I... ehh... let's say "dreamed" about similar experiences a few times and it was a powerful "dream". But being able to do it on demand would I think change how grieving works in the modern world so much! And we rely on the things that went to not actually be there for us on demand to be able to move on.

It's a beautiful idea but it's as beautiful as the fear of nuclear MAD keeping the first world safe from war in their own land, not as beautiful as a poem or a flower or a good memory.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#88

Post by corysama » Fri, Mar 25, 2022, 11:43 PM UTC

sorenjan wrote: post #44

I don't really understand why NeRFs would be particularly useful in more than a few niche cases, perhaps because I don't fully understand what they really are. My impression is that you take a bunch of photos in various places and directions, then you use those as samples of a 3D function that describes the full scene, and optimize a neural network to minimize the difference between the true light field and what's de…

If you tried to repro these results (including time & space constraints) using traditional photogrammetry, you would be sorely disappointed.

Photogrammetry is great if you have a very solid object that is not shiny or translucent at all. You get a lot of surface color micro-detail and a bit of bumpy meso-detail.

But, if something is fuzzy, hairy, or lacey or smokey you are straight-up out of luck. Don't even try.

If it is shiny, it can be difficult to capture at all --let alone capture the shine. Material capture techniques that are not "chalk sculpture" are rare, very limited and usually experimental.

NeRFs however are pretty much a photograph that you can walk around in. They have about as much structure as a photograph ;) But, that lets them not care about the mathematical definition of your scuffed-up, lacquered, iridescent, carbon fiber mirror frame and just show it as it looks from whatever angle.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#89

Post by patientplatypus » Fri, Mar 25, 2022, 11:43 PM UTC

What's the current state of research on true volumetric displays? That's what I'm excited for, although that takes less AI and more hardware, so quite a bit more difficult.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#90

Post by Tenoke » Fri, Mar 25, 2022, 11:45 PM UTC

I hope someone can take this, all the images of street view, recent images of places etc. and create a 3d environment covering as much of earth as possible to be used for an advanced Second Life or other purposes.