Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Lifecast (YC W22) – 3D video for VR

news.ycombinator.com

Post reply on HN
Print view View original

31–38 of 38 posts

Re: Launch HN: Lifecast (YC W22) – 3D video for VR

#31

Post by acgourley »

moralestapia wrote: post #27
acgourley wrote: post #3

We're working on something similar - several scene layers packed and transmitted over h265 streams and unpacked into a 3D client for 6DoF playback. Captures from something as simple as a GoPro and then our CV compares perspectives of the scene across time to reconstruct it in 3D for the encoding/transmission steps. Targeting exercise market (where we got our start) but it could go beyond it in time. Short demo: https…

No email on your profile, ese! Aside from that BitGym looks great!

Whoops - added it. Thanks!

Re: Launch HN: Lifecast (YC W22) – 3D video for VR

#33

Post by fbriggs »

glinkot wrote: post #28

This is a really worthwhile goal. I grabbed my Quest 2 and jumped onto the site. I wondered if the artifacts around areas in motion (eg the parachutist) are from compression or the 6DOF conversion process? Some 'tiling' effects on the grass areas had me wondering the same, but I imagined these were because the algorithm generates a certain size mesh even when the surface is fairly flat. What options are there for fil…

Artifacts at the edges are due to occlusions. An occlusion is a part of the scene which wasn't visible to the original camera. You see these if you move far from the original camera in VR to look behind something. This is a really hard problem for 6DOF. We've been improving the quality of occlusions over time, e.g.: * v1: https://lifecastvr.com/demo_maui.html * v2: https://lifecastvr.com/kalalea_fire.html * v3: https://lifecastvr.com/hubner4.html Version 3 now uses a 2-layer representation which has an image+depthmap for the background layer, which is drawn to fill in the occlusions. This background layer can be precomputed in a variety of different ways. For example, here is a CGI synthetic scene where we can construct the background layer perfectly: https://lifecastvr.com/liferay.html However, making up the background layer for real-world content is more challenging. We are on version 1 of that. We will improve this with machine learning in a future release. We can also substitute a "plate" 3d scene for the background in cases where the camera doesn't move. We have also experimented with using data from other frames when the camera moves. This will improve over time.

Re: Launch HN: Lifecast (YC W22) – 3D video for VR

#34

Post by soylentgraham »

fbriggs wrote: post #33
glinkot wrote: post #28

This is a really worthwhile goal. I grabbed my Quest 2 and jumped onto the site. I wondered if the artifacts around areas in motion (eg the parachutist) are from compression or the 6DOF conversion process? Some 'tiling' effects on the grass areas had me wondering the same, but I imagined these were because the algorithm generates a certain size mesh even when the surface is fairly flat. What options are there for fil…

Artifacts at the edges are due to occlusions. An occlusion is a part of the scene which wasn't visible to the original camera. You see these if you move far from the original camera in VR to look behind something. This is a really hard problem for 6DOF. We've been improving the quality of occlusions over time, e.g.: * v1: https://lifecastvr.com/demo_maui.html * v2: https://lifecastvr.com/kalalea_fire.html * v3: https…

When moving onto multiple (depth) camera setups, in-painting from old frames worked really well, even before any masking off of static vs moving content (done in realtime, for live streaming)

Re: Launch HN: Lifecast (YC W22) – 3D video for VR

#35

Post by echelon »

acgourley wrote: post #21
pedalpete wrote: post #16

Earlier quoted context omitted.

This is where we were looking to go with our "metaverse for sports" app https://ayvri.com - Blending of 3D world geometries with video and photos captured from ground view, built into 3D models. I believe this is the future of video, I was calling it "spatial media" at the time. We still operate Ayvri, but have mostly moved on to other projects.

Sometimes someone links me to Ayvri (due to our overlap here) and it's always super impressive. As the capture tools get better, and something as sturdy as a GoPro can capture stable video suitable to watch on VR, the concept might pick up.

Totally going to happen, it's just a matter of timing.

Ayvri folks, don't throw it away. Be ready to pounce on this again in the future. This idea is golden.

Superbowl 2030 will be streamed with this tech.

Re: Launch HN: Lifecast (YC W22) – 3D video for VR

#37

Post by Findeton »

pault wrote: post #18

I’ll address the elephant in the room; I think that at this time, the use case for this with the widest potential for adoption is VR porn. I would very much like to see how well this adapts to, uh, organic surfaces. I’m not joking, so please don’t downvote me. :)

200% agree.

Re: Launch HN: Lifecast (YC W22) – 3D video for VR

#38

Post by pedalpete »

echelon wrote: post #35
acgourley wrote: post #21

Earlier quoted context omitted.

Sometimes someone links me to Ayvri (due to our overlap here) and it's always super impressive. As the capture tools get better, and something as sturdy as a GoPro can capture stable video suitable to watch on VR, the concept might pick up.

Totally going to happen, it's just a matter of timing. Ayvri folks, don't throw it away. Be ready to pounce on this again in the future. This idea is golden. Superbowl 2030 will be streamed with this tech.

Thanks for the support, we still operate Ayvri, but we've moved on to a space that is perhaps better suited to or abilities and market timing in the sleep space https://soundmind.co
Post reply on HN