Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Light Field Rendering and Streaming for VR and AR [video]

on-demand.gputechconf.com

Post reply on HN
Print view View original

11–20 of 26 posts

Re: Light Field Rendering and Streaming for VR and AR [video]

#11

Post by otoy »

TD-Linux wrote: post #8
FieldSleep wrote: post #4

Earlier quoted context omitted.

I would hope not, at ~2:45 the presenter mentions working towards open formats, such as their orbx format which is open source.

Is the orbx format specification public? I can't seem to find it.

I need to download the latest version that was submitted to MPEG for the file and serial data format and get it on OTOY.com like the 1st doc (intro PDF) linked in my post above. Will share a link as soon as that's done. The container and stream are just one layer, the full node schema and module system is mapped out on docs.otoy.com for users and developers.

We're working on a flat C API for ORBX script nodes which will (we hope) leverage WASM+WEBGPU/VK to add novel functionality within the node graph itself instead of needing it to be in an arbitrary external renderer. This was how we first implements ORBX1 video in pure JavaScript back in 2013

Note that Graphs can further 'render' a scene to valid glTF/mesh and volume formats, not just texture/image output buffers.

I don't think glTF will be able to fully replace alembic/FBX soon in the schema sadly, based on last KHR discussion at siggraph (I understand the reasoning, do not going to advocate they change it); glTF is still a useful asset format for proxy volumes or baking to webgl pipelines.

Re: Light Field Rendering and Streaming for VR and AR [video]

#12

Post by lightedman »

erikpukinskis wrote: post #6

OTOY is absolutely at the forefront of digital imagery. They're like MPEG in a Kodak world, that's how different their approach is. How? 1) Light fields are 4 dimensional images, light field video is a 5 dimensional stream. This is a basic requirement for hand- or head-tracked images, like in headsets or AR devices. 2) We're just getting to the point where real time ray tracing is truly economical, and OTOY is all-in…

"Which means it can work on every device"

From OTOY.com's page: "For optimal performance and smooth interactive use, Windows 7 64bit, 8GB system RAM, a modern Quad core CPU, and a GTX brand NVidia graphics card (like GTX 560 / 570 / 580 / 590) with 1536MB VRAM or more is recommended."

Sounds like a minimum hardware spec using fairly recent hardware to me.

Re: Light Field Rendering and Streaming for VR and AR [video]

#13

Post by otoy »

ryandamm wrote: post #5
FieldSleep wrote: post #4

Earlier quoted context omitted.

I would hope not, at ~2:45 the presenter mentions working towards open formats, such as their orbx format which is open source.

Open source doesn't mean royalty free (though I have no idea if this format would be royalty-bearing). There are open source implementations of various MPEG and JPEG codecs, but they still carry a licensing fee to use them. Most people who don't use Linux don't realize this (installing Linux and trying to deal with av codecs is a quick education in the difference).

ORBX framework should be like Lua or luaJIT -i.e. MIT license. What is built or done on top of that and how it's licensed to others is up to the creator or implementer to decide, assuming they have such rights already.

Re: Light Field Rendering and Streaming for VR and AR [video]

#14

Post by otoy »

lightedman wrote: post #12
erikpukinskis wrote: post #6

OTOY is absolutely at the forefront of digital imagery. They're like MPEG in a Kodak world, that's how different their approach is. How? 1) Light fields are 4 dimensional images, light field video is a 5 dimensional stream. This is a basic requirement for hand- or head-tracked images, like in headsets or AR devices. 2) We're just getting to the point where real time ray tracing is truly economical, and OTOY is all-in…

"Which means it can work on every device" From OTOY.com's page: "For optimal performance and smooth interactive use, Windows 7 64bit, 8GB system RAM, a modern Quad core CPU, and a GTX brand NVidia graphics card (like GTX 560 / 570 / 580 / 590) with 1536MB VRAM or more is recommended." Sounds like a minimum hardware spec using fairly recent hardware to me.

For rendering an ORBX in Octane then yes (480 should be minimum), for playback 9( an ORBX package exported from Octane render job/target, HTML5 with webgl is enough for basic 360 viewing even without vr or ar

Re: Light Field Rendering and Streaming for VR and AR [video]

#16

Post by yoz »

At 13:45 he talks about:

1: streaming a Unity/Unreal game into a surface texture

2: packaging an entire Unity project into an ORBX file

So... am I understanding this right: that ORBX can contain not just a light field, but all the assets and logic for a game, compiled to LuaJIT, which the ORBX player (or orbx.js) will then play? And Unity can target this for output?

Re: Light Field Rendering and Streaming for VR and AR [video]

#18

Post by jayd16 »

I find everything I read about OTOY's stuff (seemingly purposefully) confusing.

They seem to have a render farm, light field format and renderer, and a streaming video format that they will always mix together in their demos.

"Look at these amazing renders. It runs on a phone!"*

*its actually just streaming video to the phone.

I'm pretty excited about this stuff but I keep finding myself frustrated trying to crack through OTOY's marketing to get my hands on something I can try myself.

Can someone please break my incredulity?

Re: Light Field Rendering and Streaming for VR and AR [video]

#19

Post by orbital-decay »

Findeton wrote: post #17

What about real video recording to generate real VIDEO light fields? Also: how much processing is it required to get video light fields from video recording (with an array of cameras I suppose)? I mean: does it scale?

Well, Lytro is still in business and they are doing precisely that. https://www.lytro.com/immerge

It's a true plenoptic (light-field) camera.

Re: Light Field Rendering and Streaming for VR and AR [video]

#20

Post by Findeton »

orbital-decay wrote: post #19
Findeton wrote: post #17

What about real video recording to generate real VIDEO light fields? Also: how much processing is it required to get video light fields from video recording (with an array of cameras I suppose)? I mean: does it scale?

Well, Lytro is still in business and they are doing precisely that. https://www.lytro.com/immerge It's a true plenoptic (light-field) camera.

Well, does that scale well? how much does it cost? Can a camgirl use it to "stream" (not live) her shows, for example?
Post reply on HN