Light Field Rendering and Streaming for VR and AR [video]

How?

1) Light fields are 4 dimensional images, light field video is a 5 dimensional stream. This is a basic requirement for hand- or head-tracked images, like in headsets or AR devices.

2) We're just getting to the point where real time ray tracing is truly economical, and OTOY is all-in on it. Up til now rendering has been a "bag of tricks" approach, where you try to paint sophisticated paintings on polygons. Many of these tricks fall apart when you try to do the predictive modeling required for 6DOF streaming. You see the reflections painted onto the countertop. Ray tracing actually simulates light.

3) They've fully embraced the cloud. They're offering everything they do as cloud services, which means it can work on every device, for a minimal cost, with no need for customers or users to be on the latest hardware.

4) Open formats. They're not trying to build a portal the way Oculus or Valve is, they're inventing the content pipeline and getting it integrated everywhere they can. I am skeptical any the closed content stores will win, we saw how big the web became, I think a better bet is that the metaverse will be more like the web than the App Store, and that's the bet OTOY is making.

Orbach has been working relentlessly on this vision behind the scenes. Not a lot has been coming out of the company, but I've been watching him lay the groundwork for the whole next generation of content distribution for 5 years now, and he's killing it. Release after release of core building blocks.