Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Diffusion to 3D/WebVR, in the cloud, available now

holovolo.tv

Post reply on HN
Print view View original

11–20 of 23 posts

Re: Stable Diffusion to 3D/WebVR, in the cloud, available now

#11

Post by p1mrx »

bhaney wrote: post #4

This looks... pretty terrible. The images being generated are fine, but the conversion from 2D to 3D is awful. It looks like something poorly lasso-tool'd around the subject, put it on another layer closer to the viewer, and then very poorly interpolated the space that's visible between the two layers when you look at it from an angle. Am I missing something? I feel like I've seen much better automatic 2D->3D convers…

Maybe they accidentally trained the AI to fuck up the VR180 camera projection?

The left/right sides of every image contain a different, scaled and rotated image. Some of the discontinuities are visually pretty interesting.

Re: Stable Diffusion to 3D/WebVR, in the cloud, available now

#12

Post by joewhatkins »

p1mrx wrote: post #11
bhaney wrote: post #4

This looks... pretty terrible. The images being generated are fine, but the conversion from 2D to 3D is awful. It looks like something poorly lasso-tool'd around the subject, put it on another layer closer to the viewer, and then very poorly interpolated the space that's visible between the two layers when you look at it from an angle. Am I missing something? I feel like I've seen much better automatic 2D->3D convers…

Maybe they accidentally trained the AI to fuck up the VR180 camera projection? The left/right sides of every image contain a different, scaled and rotated image. Some of the discontinuities are visually pretty interesting.

I think this is just created by running a depth prediction model on the output of stable diffusion and then inserting the relevant mesh into a 3D scene. The output of stable diffusion isn’t seamless by default, so those jumps will happen.

Re: Stable Diffusion to 3D/WebVR, in the cloud, available now

#13

Post by joewhatkins »

kingkawn wrote: post #6
bhaney wrote: post #4

This looks... pretty terrible. The images being generated are fine, but the conversion from 2D to 3D is awful. It looks like something poorly lasso-tool'd around the subject, put it on another layer closer to the viewer, and then very poorly interpolated the space that's visible between the two layers when you look at it from an angle. Am I missing something? I feel like I've seen much better automatic 2D->3D convers…

Given how 2D looked even a few years ago I’ve got high hopes for this

This is just an off the shelf img2depth model run on top of stable diffusion - I don’t think there’s a novel model or research behind this. People have been doing the same thing in colab for a while.

Re: Stable Diffusion to 3D/WebVR, in the cloud, available now

#14

Post by ajmurmann »

supermatt wrote: post #5

Im guess im not able to view the effect on desktop? Is it some kind of depth segmentation of the generated images rather than actual 3d? Maybe I need to view in a VR headset?

Theis link from the OP seems to work well on desktop: https://holovolo.tv/v/962583

That said, it looks like the ninja on the ostrich is a paper cutout that just has different parallax scrolling and you can still see the hole in the background it was cut out off.

Re: Stable Diffusion to 3D/WebVR, in the cloud, available now

#16

Post by kingkawn »

joewhatkins wrote: post #13
kingkawn wrote: post #6

Earlier quoted context omitted.

Given how 2D looked even a few years ago I’ve got high hopes for this

This is just an off the shelf img2depth model run on top of stable diffusion - I don’t think there’s a novel model or research behind this. People have been doing the same thing in colab for a while.

i dont mean this singular method in particular, just the ability to run automatic conversions to 3d or generating 3d assets without needing to hire modelers

Re: Stable Diffusion to 3D/WebVR, in the cloud, available now

#19

Post by adfm »

fbriggs wrote: post #2

"A CYBERPUNK NINJA RIDING AN OSTRICH THROUGH THE STREET OF TOKYO" https://holovolo.tv/v/962583 Today Lifecast unveils text-to-full 3D immersive environments that can be viewed in VR (e.g., Quest 2) or on 2D screens. We are doing this with a combination of Stable Diffusion and several other neural nets to make it 3D, combined with Lifecast's format for 6DOF VR photos and video. It's free to try and we do the processin…

What’s stopping you from offering a mobile stereoscopic view? There’s likely more Google Cardboard users out there than active Horizons users at this point.
Post reply on HN