Inside Magic Leap

There is a bigger problem, as well. Tracking, the glasses will need to use a lot of the GPU (and CPU) processing power for simply tracking the environment. So they don't have backgrounds, that doesn't account for much anyway, like maybe 10% of GPU power goes to that. Tracking an environment takes like an entire card at the moment.

Only by some combination of streaming and prediction; doing the rendering in the cloud, and using clever technology to hide latency.

Seems like most people here are skeptical/dubious of ML. I don't blame them there is a lot of hype. But it is quite fascinating technology. If you look seriously at the detailed technical coverage on how the technology works [1] and watch the recent Stanford lecture by Brian Schowengerdt [2], Co-Founder of Magic Leap, then I think you will see why ML is actually quite special.

Personally I am fascinated and optimistic about the potential represented by scanning fiber + waveguide display for AR and VR. I'm optimistic about the future of this tech platform far more than any other AR/VR tech out there mainly owing to form factor it affords (lightweight glasses) and the superior bio-compatibility of the light-field it produces (all day use). On principal its imagery is clearly light-years ahead of anything using a flat screen with lenses and its potential form factor could be surprisingly close to regular glasses. The image quality and form factor are huge part of why this tech could replace phone or a tv or a laptop. I am sure the first iterations will be cool but not perfect. I expect a progression much like that of the iPhone. It works, its awesome, then it keeps getting better and better and better.

IMHO this really does look like a viable platform for the future of computing for the next 10-20 years. People still buy PCs but the laptop largely replaces most uses for it other than hard-core gaming. Similarly I could see the same effect happening with screens vs ML over time. I say all this with the caveat that the form factor really is much much closer to glasses than something like HoloLense.

Here are couple things I have learned about the tech that kind of blew my mind:

* The same tech used to display the light-field can also be used to capture images of they eye for eye tracking, no extra camera needed. It is conceivable that the same scanning fiber technology could be used to dramatically miniaturize the sensors needed for SLAM inputs as well.

* The detail falloff of human vision from a small point in the center is huge. We largely piece together a detailed picture over time in our minds with quick eye movements. This means that good eye-tracking + foveated rendering largely mitigates concerns about achieving a very high perceived resolution with manageable IO.

* I assumed this kind of display would only be able to produce semitransparent ghostly images. Not so. Apparently it can block incoming light from points where it is projecting.

* It is not simple stereoscopy, the light hitting the eye is focusable at the distance the virtual object appears to exist in real space rather than at the surface of the waveguide. This is why it is so compatible with the human visual system and can be used all day as opposed to other tech. This has to do with how the visual system to point both eyes at an object in space is wired in sync with the system that focuses the lenses. No other system, to my knowledge, accommodates this linkage and thus causes strain. Watch the video to understand this more.

[1] http://uploadvr.com/magic-leap-how-it-works/ [2] https://talks.stanford.edu/brian-schowengerdt-human-sensory-...

Which Pascal? Assuming the smallest desktop Pascal, GP107 is at 768 CUDA cores. Tegra X2 Pascal has around 256 CUDA cores. They seem to be clocked similarly. The Tegra also has a lot lower memory bandwidth and stuff like that but we'll put that aside for now. We'll need the Tegra to x3 in performance. Maybe in 6? years we can get a GP107 performance in tablet form factor. The GTX 1080 has 2560 CUDA cores. That's another 3x from GP107. So maybe 12 years for ~1080 performance in tablets.

If you hold out your thumb between your eye and the sun, such that your thumb appears to just block the edges of the sun, then you will be effectively blocking the entire set of overlapping wavefronts emanating from the sun (that would have formed multiple "pixels" forming the sun image on your retina), preventing their arrival at your cornea and ultimately entering your pupil. Notice that your thumb is about about 1.5ft from your eye.

Could you emulate that with a transparent LCD display? Yes, but that display would need to be large and about 1.5ft from your eye. Not really practical for a near eye (head mounted display).

If you try to accomplish this occlusion directly with an LCD microdiplay (such as those found in many projectors) in a near eye application, at best you end up with a blackish smudge with very fuzzy edges. To see this in action, look at Figure 11.D in the following paper:

http://www.cs.unc.edu/~maimone/media/pinlights_siggraph_2014...