Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open-sourcing DeepFocus, an AI-powered system for more realistic VR images

code.fb.com

Post reply on HN
Print view View original

1–10 of 12 posts

Re: Open-sourcing DeepFocus, an AI-powered system for more realistic VR images

#5

Post by kranner »

wincy wrote: post #2

This means Oculus has this technology working. Which means foveated VR headsets are coming soon.

This seems unrelated to foveated rendering unless I'm missing something. Foveated rendering reduces the complexity of rendering in areas outside what the fovea is looking at, because they are blurry to us anyway. This adds blur to out-of-focus objects; but it seems obvious that in order to appreciate the effect you'd have to be looking at them via your fovea.

Re: Open-sourcing DeepFocus, an AI-powered system for more realistic VR images

#6

Post by russdill »

kranner wrote: post #5
wincy wrote: post #2

This means Oculus has this technology working. Which means foveated VR headsets are coming soon.

This seems unrelated to foveated rendering unless I'm missing something. Foveated rendering reduces the complexity of rendering in areas outside what the fovea is looking at, because they are blurry to us anyway. This adds blur to out-of-focus objects; but it seems obvious that in order to appreciate the effect you'd have to be looking at them via your fovea.

The two are related. In order to infer the user's focal length, you need to know what they are looking at. Which requires eye tracking, which is also a requirement of foveated rendering

Re: Open-sourcing DeepFocus, an AI-powered system for more realistic VR images

#8

Post by taneq »

IshKebab wrote: post #3
wincy wrote: post #2

This means Oculus has this technology working. Which means foveated VR headsets are coming soon.

No it doesn't. You're ignoring hardware.

Hardware meaning the headsets which were mentioned in GP post?

Re: Open-sourcing DeepFocus, an AI-powered system for more realistic VR images

#9

Post by taneq »

I really don't understand why they couldn't just use traditional blur techniques. They say:

> These fast but inaccurate methods of creating “game blur” ran counter to Half Dome’s mission, which is to faithfully reproduce the way light falls on the human retina.

... but traditional Z-based blur is no less faithful than their overall "whole screen at single shifting plane of focus depending on gaze" approach. All of computer graphics is "more to do with cinematography than realism" anyway, realism is nice but if you have to choose between "looks realistic" and "looks good" you go for "looks good" every time.

Also, as others have mentioned, getting sufficient resolution for really high quality VR basically requires foveated rendering, at which point the bits that you're blurring are, by definition, not what you're looking at (since you're rendering at lower resolution outside of the fovea) and a blur algorithm that needs four GPUs to run in realtime is a complete waste of resources.

Edit: Watched the video. Their 'Circle of Confusion' map is literally just 'focus_Z - pixel_Z'. I really don't see what deep learning adds here.

Re: Open-sourcing DeepFocus, an AI-powered system for more realistic VR images

#10

Post by boulos »

taneq wrote: post #9

I really don't understand why they couldn't just use traditional blur techniques. They say: > These fast but inaccurate methods of creating “game blur” ran counter to Half Dome’s mission, which is to faithfully reproduce the way light falls on the human retina. ... but traditional Z-based blur is no less faithful than their overall "whole screen at single shifting plane of focus depending on gaze" approach. All of co…

It's more about estimating the edges, and that the underlying depth estimate is always going to be noisy and approximate. For example, the Pixel 3 does something similar for Portrait mode [1].

Don't most modern real-time DOF techniques rely on some sort of G-buffer with primitive IDs? In the article it explicitly says they wanted to just work with RGBZ so that existing titles can just be post-process blurred.

Finally, I think the time required is probably closer to 50ms per frame per GPU. If you assume that the content was running at about 60 fps (16ms/frame) on a single GPU, and linearly sped up via magic to 240 fps (4 and change ms), then the delta to 60 fps is 12 ms across four GPUs or about 48 ms on a single GPU.

[1] https://ai.googleblog.com/2018/11/learning-to-predict-depth-...

Post reply on HN