Live data from Hacker News

Meta Quest Pro – Bad AR Passthrough

kguttag.com

61–70 of 216 posts

Re: Meta Quest Pro – Bad AR Passthrough

#61

My Quest 2 is 99% used to play 5 on 5 Onward VR. To all you Onward PC users, my apologies. I thought I was done with video games but it's fantastic. The experience reminds of showing up to public basketball courts during the summer. You get on a team with some random people. Sometimes you're the best on the court, sometimes you're the worst. By I love the Quest 2 and would replace it if my got broken simply to keep p…

How accommodating is the community to new players? Seems like the type of game that would attract a…serious player base.

Re: Meta Quest Pro – Bad AR Passthrough

#62
post #44

(Edit: the tearing is not visible through the device, so quite a bit less damning than I thought.) The tear and color lag on this picture of a moving chart is damning: https://i0.wp.com/kguttag.com/wp-content/uploads/2023/01/MQP... As is this quote: > I also ran a small experiment where I sent a camera flash off in a dimly lit room and measured the delay with light sensors using one sensor to trigger an oscilloscope’…

The tear is an artifact with how the screen capture works. The tear is not actually sent to the display. People who capture Quest footage using scrcpy are well aware of this downside compared to using the built in recording feature which doesn't have the same tearing issue.

[deleted]

Re: Meta Quest Pro – Bad AR Passthrough

#63
post #26

How good is the resolution? Can you already use the device as a monitor-replacement? (I.e., coding behind a virtual monitor on an exotic location, while you're just at home behind the desk)

It is good enough that you can imagine the possibility of using it. Not good enough to actually use it like this (IMO/IME- There’s stories here of people doing this).

A few generations and it’ll be doable.

Re: Meta Quest Pro – Bad AR Passthrough

#65
post #12

I don’t think people fully grasp how much power and compute it takes to record and process video (me included). let me know how long that canon r5 can record for continuously, and how many batteries it’ll go through for that recording time. It wasn’t that long ago that it was taxing for desktop computers to play a 4k video file.

With cameras the problem isn't the power or compute. It's the heat dissipation. Cameras like the Sigma FP are built around the heat sink and have vents all around to dissipate the heat during long recording sessions. Most typical DSLRs simply have recording limits e.g. Canon R6 = 30 mins. Pretty big issue for a device they are aiming to be around your head for hours at a time.

Well, that's not exactly right -they're two sides of a coin. twice the pixels equals twice the power at almost every level. The pixel level, the image signal processor level, the processor, and storage.

Re: Meta Quest Pro – Bad AR Passthrough

#66
post #45

The author persistently conflates AR and MR throughout the posting, evaluating the Quest Pro as an AR device but the concluding at the end it's a terrible "MR" device because it failed at their AR tests. It seems to suggest they really think these terms are synonymous but they aren't. While AR aims to enable display and interaction with the real world but with augmentations, MR aims to bring virtual objects into the…

This blog post is written by Karl Guttag who is someone who is interested in AR, but not very interested in VR. The reason he is coming at it from an AR angle is that is what kind of device he is looking for. If you look at the rest of his blog you can see many articles related to AR.

Re: Meta Quest Pro – Bad AR Passthrough

#67
post #26

How good is the resolution? Can you already use the device as a monitor-replacement? (I.e., coding behind a virtual monitor on an exotic location, while you're just at home behind the desk)

It's just usable as a monitor replacement, but it would be a big stretch to say it's "better". If you have even a cheap a monitor available you'd nearly always be better off actually using it. The scenario where it's better are going to be specific use cases like:

- it's helpful to have it be gigantic - bigger than a real monitor could ever be

- you have a real need for more monitors than would be practical to buy or position in your work environment

- you will be highly mobile and moving around so much that it's going to be better to take a headset with you than try to get monitors everwhere

- you really would like to collaborate with someone else who also has a Quest Pro, in which case you can work together without being physically together

- you highly benefit from / enjoy working in alternative environments (eg: being highly focused, free of distractions, relaxed, surrounded by beatiful scenery etc)

I actually find the latter point is the biggest reason I do it. Even when I'm surrounded by distractions I can go into a completely calm virtual environment and get a crap load of work done.

Re: Meta Quest Pro – Bad AR Passthrough

#68
post #45

The author persistently conflates AR and MR throughout the posting, evaluating the Quest Pro as an AR device but the concluding at the end it's a terrible "MR" device because it failed at their AR tests. It seems to suggest they really think these terms are synonymous but they aren't. While AR aims to enable display and interaction with the real world but with augmentations, MR aims to bring virtual objects into the…

As someone close to this in the industry, I definitely agree.

I also really don't know why he decided to deemphasize the perspective and depth correctness so much. He mentions it here:

> In this case, they were willing to sacrifice image quality to try to make the position of things in the real world agree with where virtual objects appear. To some degree, they have accomplished this goal. But the image quality and level of distortion, particularly of “close things,” which includes the user’s hands, is so bad that it seems like a pyrrhic victory.

I don't think this is even close to capturing how important depth and perspective correct passthrough is.

Reprojecting the passthrough image onto a 3D representation of the world mesh to reconstruct a perspective-correct view is the difference between a novelty that quickly gives people headaches and something that people can actually wear and look through for an extended period of time.

Varjo, as a counterexample, uses incredibly high-resolution cameras for their passthrough. The image quality is excellent, text is readable, contrast is good, etc. However, they make no effort to reproject their passthrough in terms of depth reconstruction. The result is a passthrough image that is very sharp, but is instantly, painfully, nauseatingly uncomfortable when walking around or looking at closeup objects alongside a distant background.

The importance of depth-correct passthrough reprojection (essentially, spacewarp using the depth info of the scene reconstruction mesh) absolutely cannot be understated and is a make or break for general adoption of any MR device. Karl is doing the industry a disservice with this article.

Re: Meta Quest Pro – Bad AR Passthrough

#69
post #44

(Edit: the tearing is not visible through the device, so quite a bit less damning than I thought.) The tear and color lag on this picture of a moving chart is damning: https://i0.wp.com/kguttag.com/wp-content/uploads/2023/01/MQP... As is this quote: > I also ran a small experiment where I sent a camera flash off in a dimly lit room and measured the delay with light sensors using one sensor to trigger an oscilloscope’…

The tear is an artifact with how the screen capture works. The tear is not actually sent to the display. People who capture Quest footage using scrcpy are well aware of this downside compared to using the built in recording feature which doesn't have the same tearing issue.

Thanks, I missed that. Posted an edit at the top of my comment.

Re: Meta Quest Pro – Bad AR Passthrough

#70
post #68
post #45

The author persistently conflates AR and MR throughout the posting, evaluating the Quest Pro as an AR device but the concluding at the end it's a terrible "MR" device because it failed at their AR tests. It seems to suggest they really think these terms are synonymous but they aren't. While AR aims to enable display and interaction with the real world but with augmentations, MR aims to bring virtual objects into the…

As someone close to this in the industry, I definitely agree. I also really don't know why he decided to deemphasize the perspective and depth correctness so much. He mentions it here: > In this case, they were willing to sacrifice image quality to try to make the position of things in the real world agree with where virtual objects appear. To some degree, they have accomplished this goal. But the image quality and l…

yes I think the success of the reprojection is a much under appreciated achievement. The author repeatedly questions their claim of "seamless" but its one of the most stunning aspects that you can, for example, see your real arm outside the headset exactly map onto your virtual arm and hand inside it, as do other objects that bridge from external reality to the virtual reality view. This "seamless" nature of it is really impressive when you consider that the whole view is actually being completely reprojected and not just displayed natively from the camera feeds.
Post reply on HN