Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia Research Turns 2D Photos into 3D Scenes

blogs.nvidia.com

Post reply on HN
Print view View original

41–50 of 239 posts

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#41

Post by doliveira »

c4wrd wrote: post #10
simsla wrote: post #5

Earlier quoted context omitted.

> plus data on the camera angles they were taken from Doesn't seem like much of a stretch to determine the angles as well. E.g. a semi brute forced way with GANs

I've spent a lot of time thinking about this (i.e. taking a video and creating a 3D scene) and I don't think that it is feasible in most cases to have good accuracy. If you need to infer the angle, you need make a lot of biased assumptions about things like velocity, position, etc., of the camera and even if you were 99.9% accurate, that 0.1% inaccuracy is compounded over time. Now I'm not saying it's not possible, b…

> even if you were 99.9% accurate, that 0.1% inaccuracy is compounded over time

Not really, with SLAM there are various algorithms to keep inaccuracy in check. Basically it works by a feedback loop of guessing an estimate for position and then updating it using landmarks.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#42

Post by ksec »

Nvidia is really turning into an AI powerhouse. The moat around CUDA, and how those target customer aren't as stringent about budget, especially when the hardware cost is tiny compare to what they do.

I wonder if they could reach a trillion market cap.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#43

Post by danamit »

I am kinda skeptical, AI demos are impressive but the real world results are underwhelming.

How much it resources it takes to generate images like that? is this the most ideal situation?

Can you take images from the web and based on metadata make a better street view?

With all this AI where is one accessible translation service? or even an accent-adjusting service? or just good auto-subtitles?

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#44

Post by sorenjan »

I don't really understand why NeRFs would be particularly useful in more than a few niche cases, perhaps because I don't fully understand what they really are.

My impression is that you take a bunch of photos in various places and directions, then you use those as samples of a 3D function that describes the full scene, and optimize a neural network to minimize the difference between the true light field and what's described by the network. An approximation of the actual function, that fits the training data. The millions of coefficients are seen as a black box that somehow describes the scene when combined in a certain way, I guess mapping a camera pose to a rendered image? But why would that be better than some other data structure, like a mesh, a point cloud, or signed distance field, where you have the scene as structured data you can reason about? What happens if you want to animate part of a NeRF, or crop it, or change it in any way? Do you have to throw away all trained coefficients and start again from training data?

Can you use this method as a part of a more traditional photogrammetry pipeline and extract the result as a regular mesh? Nvidia seems to suggest that NeRFs are in some way better than meshes, but according to my flawed understanding they just seem unwieldy.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#45

Post by rilezg »

baron816 wrote: post #31

It would be really great to recreate loved ones after they have past in some sort of digital space. As I’ve gotten older, and my parents get older as well, I’ve been thinking more about what my life will be like in old age (and beyond too). I’ve also been thinking what I would want “heaven” to be. Eternal life doesn’t appeal to me much. Imagine living a quadrillion years. Even as a god, that would be miserable. That…

This reminds me a lot of Black Mirror season 2 episode 1.

Always good to treasure the time we are given.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#46

Post by olladecarne »

baron816 wrote: post #31

It would be really great to recreate loved ones after they have past in some sort of digital space. As I’ve gotten older, and my parents get older as well, I’ve been thinking more about what my life will be like in old age (and beyond too). I’ve also been thinking what I would want “heaven” to be. Eternal life doesn’t appeal to me much. Imagine living a quadrillion years. Even as a god, that would be miserable. That…

There's so many things we invent with good intentions but in the end go terribly wrong and I think this is one of those things. I think it's ok to mourn and remember the past, but moving on and accepting reality is important to a healthy life.

Let's be real though, the startup that makes this but appeals to our worst instincts make bank. I can't imagine how much more messed up future generations will be as we keep making more dangerous technology that appeals to our primal instincts.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#47

Post by lowdest »

baron816 wrote: post #31

It would be really great to recreate loved ones after they have past in some sort of digital space. As I’ve gotten older, and my parents get older as well, I’ve been thinking more about what my life will be like in old age (and beyond too). I’ve also been thinking what I would want “heaven” to be. Eternal life doesn’t appeal to me much. Imagine living a quadrillion years. Even as a god, that would be miserable. That…

>Imagine living a quadrillion years. Even as a god, that would be miserable.

This seems very subjective, I don't agree at all.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#48

Post by the_mar »

rilezg wrote: post #45
baron816 wrote: post #31

It would be really great to recreate loved ones after they have past in some sort of digital space. As I’ve gotten older, and my parents get older as well, I’ve been thinking more about what my life will be like in old age (and beyond too). I’ve also been thinking what I would want “heaven” to be. Eternal life doesn’t appeal to me much. Imagine living a quadrillion years. Even as a god, that would be miserable. That…

This reminds me a lot of Black Mirror season 2 episode 1. Always good to treasure the time we are given.

and then you have to put your loved one's in the attic

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#49

Post by teaearlgraycold »

tomatowurst wrote: post #29
teaearlgraycold wrote: post #23

Earlier quoted context omitted.

RIP photo-realistic modelers

hmmm well I still think they will be in demand for the same reason software developers will be not automated away. NeRF is really mind boggling good but there are still artifacts, and something that modelers have a good eye for. Having said that, it might be the end for any junior type of roles. Same reason that github copilot really takes a bite of the need to have a junior developer. I'm very curious what will happ…

Maybe we'll have people spend their time building IRL sculptures and spaces to get digitized.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#50

Post by ModernMech »

sennight wrote: post #18

I know that taste in comedy is seasonal (yes, there were a people in a time that thought vaudeville was the cat's pajamas), but has anyone ever greeted a pun with anything other than a pained sigh?

Watch Bob's Burgers. The whole show is basically puns. I chuckle.
Post reply on HN