Live data from Hacker News

Nvidia Research Turns 2D Photos into 3D Scenes

blogs.nvidia.com

161–170 of 239 posts

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#161

Earlier quoted context omitted.

Bs. People do different things. But nobody can sit around wallowing in the VR. Having seen what death of my cousin and grand parents did to the family, moving on helped every one heal in a way. VR would have been a torture to live in.

Perhaps starting your comment with "bs" is not a surefire way of getting your thoughts considered seriously..

Sigh. Point taken. You are right.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#162

Earlier quoted context omitted.

Bs. People do different things. But nobody can sit around wallowing in the VR. Having seen what death of my cousin and grand parents did to the family, moving on helped every one heal in a way. VR would have been a torture to live in.

You are taking the worst possible interpretation of my comment. I was responding to literally the aspect of the GP that said "mourn.... move on" It is a factual matter, easily discovered through simple searches, that non-Western cultures often take a different approach. Including, literally, ancestor worship. I am not making a judgement on which is better. I believe there are multiple healthy ways of dealing with gri…

I dont know how a grieving person can maintain distance from vr when the memories of their loved ines are tied. Reminds me so much of the movie Reminiscence

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#163
post #109

Earlier quoted context omitted.

My gut reaction to 'recreating loved ones in VR' is that it would be torturous more than soothing. Once a person dies, they're gone. The world is different, and nothing will bring them back. Spending time with a simulacrum isn't really spending time with a loved one.

Don't knock it til you try it? I mean, already in recent years people have made some very low fidelity 'resurrections' and gotten some measure of comfort from it, never mind the many years of history of people who visit gravestones to 'chat' (some even believing they get replies to some extent or another). When markov chains were hip, "talk with Charles Dickens!" (play with a markov model trained on his works) was at…

I suppose... but I'll always know that it's some sort of cheap trick to make me think they're still here when really their consciousness ceased.

Will the simulacrum age? Will it change? Will it ever surprise me or intrigue me? And if it does, is it something the dead person truly would have done?

It's sort of like a photograph; a photograph seems to capture reality but all it shows is some abstraction of a physical reality at one point in time. The photograph tells me nothing about the current reality of anything depicted within.

A simulacrum gives me the person I knew, when they died, and that's it. Perhaps comforting and interesting, but ultimately unfulfilling and unsettling I would imagine.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#164
post #43

I am kinda skeptical, AI demos are impressive but the real world results are underwhelming. How much it resources it takes to generate images like that? is this the most ideal situation? Can you take images from the web and based on metadata make a better street view? With all this AI where is one accessible translation service? or even an accent-adjusting service? or just good auto-subtitles?

This is essentially like a 3D JPEG, but instead of modelling the image with Discrete Cosine Transform (DCT) they use a neural net. So the neural net itself will learn just that one image and be able to reproduce it point by point, from various angles. A whole network for just one example, and an innovative way to look at what a neural network can be.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#165
post #73

as someone who works in both AI and filmmaking, I remember losing my mind when this paper was first released a few weeks ago. It's absolute insanity what the folks at Nvidia have managed to accomplish in such a short time. The paper itself[0] is quite dense, but I recommend reading it -- they had to pull some fancy tricks to get performance to be as good as it is! [0] https://nvlabs.github.io/instant-ngp/

Quoted post unavailable.

No, really, it's a cool application of neural nets. It was unexpected when it first came up and it took a whole day to learn a scene, but a couple of years later it can be done in seconds.

I think the cool part is that a neural net can learn to produce (R,G,B) from (X,Y,Z,angle) by using a clever encoding trick with sin() and cos() for the input coordinates. And the fact that a neural net can be a frigging JPEG in 3D.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#166
post #73

as someone who works in both AI and filmmaking, I remember losing my mind when this paper was first released a few weeks ago. It's absolute insanity what the folks at Nvidia have managed to accomplish in such a short time. The paper itself[0] is quite dense, but I recommend reading it -- they had to pull some fancy tricks to get performance to be as good as it is! [0] https://nvlabs.github.io/instant-ngp/

Quoted post unavailable.

You're missing this work in the context of the field as a whole. Labs have been releasing papers boasting 2-4x speedups and getting them published at conferences, and then this group comes in and speeds up the original by 1000x. That's a huge leap in capability.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#167
post #55

Earlier quoted context omitted.

You’ll find this is true of all NeRFs if you spend time playing around with them. If a NeRF is trying to render part of an object that wasn’t observered in the input images, it’s going to look strange, since it’s ultimately just guessing at the appearance. The NVidia example in the link has the benefit of focusing on a single entity that’s centered in all of the input photographs - the effect is much more pronounced…

I know when doing typical 2D video based rotoscoping it is possible to use frames from before/after the current frame to see data that is being blocked in the current frame. It's also common in restoration when removing scrathes/hair in the gate/etc. To that end, I wonder if exporting a similar bit of video from that same path exported as stills would be enough to generate the 3D version.

The nerf already does this - the problem is when part of the object doesn’t appear in any of the frames.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#168
post #99

Earlier quoted context omitted.

Self-driving car companies are already working on this. Check out: https://waymo.com/research/block-nerf/ NERF is a very active research area, and the progress from 2020 to now has been nothing sort of astonishing. In 5 years, I expect there to be fully generative NERF's in research i.e describe a scene, and a NN produces a full 3d scene that can you interact with.

Can it be done with Google's existing Street View data?

Can be done yes. Is it the best dataset to do what you describe, no.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#169
post #42

Nvidia is really turning into an AI powerhouse. The moat around CUDA, and how those target customer aren't as stringent about budget, especially when the hardware cost is tiny compare to what they do. I wonder if they could reach a trillion market cap.

it's not a matter of "if", but "when" they will reach a trillion dollar market cap.

Well I think that is a little optimistic in the near terms. Considering there has never been a Semi-Conductor player reaching that milestone without a Fab. So Nvidia will be a first. At current P/E of 70, Semiconductor industry average is only ~30. Realistically Nvidia will need to triple their revenue at a fair P/E. I could see that in Data Center, reaching $30B revenue per year within next ~5 years. And this is already larger than Intel's DataCenter record revenue. We are still $30B short coming from Gaming and Professional Visualisation. Intel already has their GPU play ready in 2022. ( Assuming it is competitive. ). i.e Nvidia will need to find another massive market to conquer to reach that Trillion Market Cap status.

Re: Nvidia Research Turns 2D Photos into 3D Scenes

#170
post #138

Earlier quoted context omitted.

My gut reaction to 'recreating loved ones in VR' is that it would be torturous more than soothing. Once a person dies, they're gone. The world is different, and nothing will bring them back. Spending time with a simulacrum isn't really spending time with a loved one.

> it would be torturous more than soothing Relevant Asimov story: https://en.wikipedia.org/wiki/The_Dead_Past

I was going to suggest the HBO film "Reminiscence", which was panned as mediocre but I liked it for being Noir and somewhat original. It features a machine that can replay your own memories and record them for others -- useful for investigating crime, but otherwise highly addictive to the nostalgic.

Reading about The Dead Past tho, there's a good deal of overlap with the plot device of "Devs", a periscope into deep history, but which Nick Offerman's character uses to re-live moments of his daughter's short life, as Asimov's protagonist does.

Post reply on HN