Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta Quest Pro – Bad AR Passthrough

kguttag.com

Post reply on HN
Print view View original

71–80 of 216 posts

Re: Meta Quest Pro – Bad AR Passthrough

#71

Post by RandallBrown »

adastra22 wrote: post #37
sdrinf wrote: post #16

The primary use case for passthrough is not AR, it was not intended to be projecting stuff over room interior, and as such, it's crap at a thing that it is explicitely not optimized for. The primary use case for passthrough is bringing stuff into VR, specifically your hands (when used in hand tracking mode), and your keyboard. Your monitor is projected into VR by being hooked to your headset. Any other function's uti…

Source? Other “Pro” models like HoloLens are explicitly AR-only. What Pro use cases are there for VR?

I figured the "Pro" in Meta Quest just means it's more powerful than the quest 2...

Re: Meta Quest Pro – Bad AR Passthrough

#72

Post by kakali »

threeseed wrote: post #22
kart23 wrote: post #12

I don’t think people fully grasp how much power and compute it takes to record and process video (me included). let me know how long that canon r5 can record for continuously, and how many batteries it’ll go through for that recording time. It wasn’t that long ago that it was taxing for desktop computers to play a 4k video file.

With cameras the problem isn't the power or compute. It's the heat dissipation. Cameras like the Sigma FP are built around the heat sink and have vents all around to dissipate the heat during long recording sessions. Most typical DSLRs simply have recording limits e.g. Canon R6 = 30 mins. Pretty big issue for a device they are aiming to be around your head for hours at a time.

That is a valid problem, but with MR pass through there is also a compute issue. The cameras are not in the same location as the users eyes. They have a wider parallax than the users eyes and are forward several inches. There is no simple transformation to correct the video to have the same perspective as the user. Instead a 3d representation of the environment must be created, camera imagery projected on to it, and then rendered to the position of the user’s eye. All of this done as fast as possible and highest rate possible. This explodes in compute with increasing resolutions. MQP is doing this with 3 cameras, color compositing, and without the aide of a depth camera.

Re: Meta Quest Pro – Bad AR Passthrough

#73

Post by georgeecollins »

zmmmmm wrote: post #45

The author persistently conflates AR and MR throughout the posting, evaluating the Quest Pro as an AR device but the concluding at the end it's a terrible "MR" device because it failed at their AR tests. It seems to suggest they really think these terms are synonymous but they aren't. While AR aims to enable display and interaction with the real world but with augmentations, MR aims to bring virtual objects into the…

I also think there is a lot to be said for a VR headset with even mediocre pass through. I find VR very engaging and I think it can be very useful -- you can have the screen space of many monitors if the headset has good enough resolution (not good enough on a quest), plus there is a ton you can do with modeling and visualization.

But it is a big turn off to have no visual idea of what is going on around you. Just to be able to press a button and be able to see that there is someone standing around you, or you are too close to a couch, is a big win for a VR headset in my opinion. Obviously the visual fidelity will improve in the display and the pass through. But just starting to get some pass through is very nice.

Re: Meta Quest Pro – Bad AR Passthrough

#74

Post by [deleted] »

pia32 wrote: post #38
Mizza wrote: post #25

I've heard that some of the AR stuff got nerfed because of the ability of IR cameras to see through some fabrics and Facebook not wanting to have those headlines in the newspapers. I just heard it as a rumour though, perhaps an insider with a throwaway account can "shed some light".

While I don't have insider info, I highly doubt this. The Quest 2 (which I own) uses monochrome cameras without IR cut filters, and they do passthrough with those. So any problems with privacy would have been found already with the current device.

[deleted]

Re: Meta Quest Pro – Bad AR Passthrough

#75

Post by kakali »

charcircuit wrote: post #66
zmmmmm wrote: post #45

The author persistently conflates AR and MR throughout the posting, evaluating the Quest Pro as an AR device but the concluding at the end it's a terrible "MR" device because it failed at their AR tests. It seems to suggest they really think these terms are synonymous but they aren't. While AR aims to enable display and interaction with the real world but with augmentations, MR aims to bring virtual objects into the…

This blog post is written by Karl Guttag who is someone who is interested in AR, but not very interested in VR. The reason he is coming at it from an AR angle is that is what kind of device he is looking for. If you look at the rest of his blog you can see many articles related to AR.

That’s fine, but it’s okay to call out the author’s bias. They contributed wonderful tests of the limitations of the headset. But the headset is only a failure for the authors use case. (AR)

Re: Meta Quest Pro – Bad AR Passthrough

#76

Post by Zee2 »

zmmmmm wrote: post #70
Zee2 wrote: post #68

Earlier quoted context omitted.

As someone close to this in the industry, I definitely agree. I also really don't know why he decided to deemphasize the perspective and depth correctness so much. He mentions it here: > In this case, they were willing to sacrifice image quality to try to make the position of things in the real world agree with where virtual objects appear. To some degree, they have accomplished this goal. But the image quality and l…

yes I think the success of the reprojection is a much under appreciated achievement. The author repeatedly questions their claim of "seamless" but its one of the most stunning aspects that you can, for example, see your real arm outside the headset exactly map onto your virtual arm and hand inside it, as do other objects that bridge from external reality to the virtual reality view. This "seamless" nature of it is re…

Yeah, I really don't think people understand how critical the reprojection is. A lot of folks, apparently Karl Guttag included, think that the resolution of the passthrough is the most important factor for usefulness. And/or that the resolution of the cameras is even directly related to the quality of the passthrough to begin with! The pixel count is just one of the dozens of quality metrics, and is arguably actually one of the less important ones.

I also can't tell if KGuttag believes that you can simply pass the video feed through to the user and not have an absolute usability disaster. His paragraph about the "pyrrhic victory" sort of implies that not preferring depth-correct scene reprojection was an option. In my experience, that is absolutely non-negotiable in terms of shipping a device that's usable to the majority of consumers.

Another reason why folks also don't understand why passthrough is so energy intensive. You're also running an entire additional pass of steresocopic spacewarp on what is essentially four different camera feeds, on an incredibly high resolution scene reconstruction/SLAM mesh.

Re: Meta Quest Pro – Bad AR Passthrough

#77

Post by mardifoufs »

edgefield wrote: post #21

A new VR arcade opened up in my area and I visited with my son. They were using HTC Vive headsets, which retail for $1,000+. It’s the first time I really used VR, especially in a couch coop mode setup. The following struck me: (1) There is no killer, must have game for VR. Beat Saber seems to be the most popular and mainstream game but not nearly engaging enough to convince the mass market to buy one or more VR heads…

Was it the original HTC Vive? Or even the pro? Because both of those are pretty old by now, with even the pro being a pretty minor upgrade to the original that came out in 2015. I'm pretty sure the hardware improved a lot, and having used the quest 2 (just once or twice at the office, not really into VR/gaming in general) it's a massively different experience from the older vive.

Re: Meta Quest Pro – Bad AR Passthrough

#78

Post by charcircuit »

kakali wrote: post #75
charcircuit wrote: post #66

Earlier quoted context omitted.

This blog post is written by Karl Guttag who is someone who is interested in AR, but not very interested in VR. The reason he is coming at it from an AR angle is that is what kind of device he is looking for. If you look at the rest of his blog you can see many articles related to AR.

That’s fine, but it’s okay to call out the author’s bias. They contributed wonderful tests of the limitations of the headset. But the headset is only a failure for the authors use case. (AR)

I agree. If you know the author the bias is clear, but it's okay to point it out for people who are not familiar with him.

Re: Meta Quest Pro – Bad AR Passthrough

#79

Post by semireg »

I’m not a AR/VR bull but I think Apple’s position is extremely underestimated.

It’s almost like every Apple product has been developed and refined for the future AR/VR moment. Custom silicon. Fast camera HDR. LiDAR. People/scene/room/boundary detection. AirPod Max fit and finish. Spatial audio. 50 other things…

All in the service of AR/VR.

Post reply on HN