Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Glasses-free 3D using webcam head tracking

assetstore.unity.com

Post reply on HN
Print view View original

51–60 of 110 posts

Re: Glasses-free 3D using webcam head tracking

#51

Post by Wowfunhappy »

president_zippy wrote: post #8
il_nets wrote: post #2

Hi HN, I'm Sten, one of the creators. We built this because we wanted 3D experiences without needing a VR headset. The approach: use your webcam to track where you're looking, and adjust the 3D perspective to match. Demo: https://portality.io/dragoncourtyard/ (Allow camera, move your head left/right) It creates motion parallax - the same depth cue your brain uses when you look around a real object. Feels like looking…

It was definitely useful and appreciated on the "New" Nintendo 3DS XL, which also used a camera to track your eye movements and adjust the divergence accordingly. I hate the fact that Nintendo abandoned this technology because experiencing Ocarina of Time and Star Fox 64 in 3D was world-changing to me. I'd say I'm not the only one who misses this technology in games, because a used New 3DS XL costs at least $200 on e…

The 3DS is different, it's using a lenticular screen so your eyes actually see different images! The eye tracking allows it to work even if the position of your eyes changes (ie because you moved your head).

Presumably developers could have combined this with parallax head tracking for an even stronger effect when you move your head (or the console), but as far as I know no one did.

Re: Glasses-free 3D using webcam head tracking

#52

Post by Wowfunhappy »

nine_k wrote: post #45
il_nets wrote: post #2

Hi HN, I'm Sten, one of the creators. We built this because we wanted 3D experiences without needing a VR headset. The approach: use your webcam to track where you're looking, and adjust the 3D perspective to match. Demo: https://portality.io/dragoncourtyard/ (Allow camera, move your head left/right) It creates motion parallax - the same depth cue your brain uses when you look around a real object. Feels like looking…

This is fun! But I see it showing me a close-up view (smaller FOV) if I move my head back, and a wider view (larger FOV) if I bring my head closer. This is the opposite to what I'd expect: if I bring my head nearer to the screen, I should see a more detail, closer (narrower FOV).

Your expectation doesn't match real life! Try it with a window. If you bring your head closer to the window, your FOV increases, ie you can see more of the scene outside the window.

Re: Glasses-free 3D using webcam head tracking

#53

Post by hex4def6 »

quitit wrote: post #18

Reminds me of the Amazon Fire phone which featured similar tech prominently. "Dynamic perspective". https://www.youtube.com/watch?v=6trOg2IK2Zg

Tangentially involved in that project.

One of the big issues with that phone was that in order to do dynamic perspective, you're having to run a 3D render at 60fps constantly. That's a huge power hog, and prevents you from doing many of the power savings techniques you otherwise could on a normal phone -- shutting down the GPU, reduced refresh rate, heck, even RAM backed displays.

Re: Glasses-free 3D using webcam head tracking

#54

Post by marcodiego »

il_nets wrote: post #2

Hi HN, I'm Sten, one of the creators. We built this because we wanted 3D experiences without needing a VR headset. The approach: use your webcam to track where you're looking, and adjust the 3D perspective to match. Demo: https://portality.io/dragoncourtyard/ (Allow camera, move your head left/right) It creates motion parallax - the same depth cue your brain uses when you look around a real object. Feels like looking…

How about contributing that to Godot?

Re: Glasses-free 3D using webcam head tracking

#55

Post by trenchpilgrim »

LordHeini wrote: post #21
il_nets wrote: post #2

Hi HN, I'm Sten, one of the creators. We built this because we wanted 3D experiences without needing a VR headset. The approach: use your webcam to track where you're looking, and adjust the 3D perspective to match. Demo: https://portality.io/dragoncourtyard/ (Allow camera, move your head left/right) It creates motion parallax - the same depth cue your brain uses when you look around a real object. Feels like looking…

The obvious use case would be to replace the clunky head tracking systems which are often used in simulator games. Systems like trackir, which require dedicated hardware.

There's already a wide variety of Opentrack plugins that use everything from off the shelf webcams to DIY infrared trackers to an iPhone app and FaceID/AirPods.

Re: Glasses-free 3D using webcam head tracking

#56

Post by regularfry »

Ninjinka wrote: post #49

Calling this 3D is a stretch, it does not make things appear like they're coming out of the screen. That requires having a way to show something different to each eye, something not possible with a standard display. I did see a demo of 3D without glasses on a full monitor that DID make it look like it was coming out of the screen at CES, it requires a $3000 monitor though: https://www.3dgamemarket.net/content/32-4k-g…

Not strictly true. Out past a certain distance your brain uses parallax for depth cues because the difference between each of the eye's images is too small. It's 3d, just not stereoscopic 3d.

Re: Glasses-free 3D using webcam head tracking

#57

Post by nine_k »

Wowfunhappy wrote: post #52
nine_k wrote: post #45

Earlier quoted context omitted.

This is fun! But I see it showing me a close-up view (smaller FOV) if I move my head back, and a wider view (larger FOV) if I bring my head closer. This is the opposite to what I'd expect: if I bring my head nearer to the screen, I should see a more detail, closer (narrower FOV).

Your expectation doesn't match real life! Try it with a window. If you bring your head closer to the window, your FOV increases, ie you can see more of the scene outside the window.

But there is no window. There is a box with a dragon in it. When I bring my eyes closer to the box, I expect to see more of the box, and more of the dragon. You can try it with any real box; substitute any appropriate object for the dragon. Pick your favorite 3D environment that features walls and dragons; notice how both grow, covering more of your FOV. This is what I expect to see as I draw closer to the screen.

Re: Glasses-free 3D using webcam head tracking

#58

Post by zipy124 »

il_nets wrote: post #2

Hi HN, I'm Sten, one of the creators. We built this because we wanted 3D experiences without needing a VR headset. The approach: use your webcam to track where you're looking, and adjust the 3D perspective to match. Demo: https://portality.io/dragoncourtyard/ (Allow camera, move your head left/right) It creates motion parallax - the same depth cue your brain uses when you look around a real object. Feels like looking…

The latency is unfortunately bad enough that it prevents me from getting any depth illusion :( I've seen other implementations where it works really well though.

Re: Glasses-free 3D using webcam head tracking

#59

Post by jeffhuys »

nine_k wrote: post #57
Wowfunhappy wrote: post #52

Earlier quoted context omitted.

Your expectation doesn't match real life! Try it with a window. If you bring your head closer to the window, your FOV increases, ie you can see more of the scene outside the window.

But there is no window. There is a box with a dragon in it. When I bring my eyes closer to the box, I expect to see more of the box, and more of the dragon. You can try it with any real box; substitute any appropriate object for the dragon. Pick your favorite 3D environment that features walls and dragons; notice how both grow, covering more of your FOV. This is what I expect to see as I draw closer to the screen.

The screen is the window.

Re: Glasses-free 3D using webcam head tracking

#60

Post by bangaladore »

richrichardsson wrote: post #20

> This content is hosted by a third party provider that does not allow video views without acceptance of Targeting Cookies. Please set your cookie preferences for Targeting Cookies to yes if you wish to view videos from these providers. OK, that's a no then.

The funny thing is I'm unable to navigate the UI to see the video. I disabled (enabled) targeted cookies, and it still doesn't show.
Post reply on HN