Jump – Experiences like you're actually there • Hacker News Board

Re: Jump – Experiences like you're actually there

#121

Post by drcross » Thu, May 28, 2015, 11:48 PM UTC

Arjuna wrote: post #58

devindotcom wrote: post #22

I love all the experimentation in this area but I'm afraid systems like this are going to fall into a sort of uncanny valley of VR. Your brain is incredibly good at picking out the discrepancies between the real world and a high-resolution 120fps image with surround sound - just as it is incredibly good at recognizing objects and motions masquerading as human. It's like going from 2D/24FPS to 3D/48FPS. You'd think it…

You might be interested in this talk that Michael Abrash gave about 2 months ago at Facebook F8 [1]. It starts at around 00:42:20. It is a fascinating talk where he provides a number of interesting demonstrations regarding how the brain interprets information, including depth perception, color interpretation, movement, auditory cues [2] etc. Highly recommended. [1] https://www.youtube.com/watch?v=UDu-cnXI8E8&t=2540 […

I found that talk to be juvenile. If I want to look at optical illusions I'll google search them myself.

Re: Jump – Experiences like you're actually there

#122

Post by drcross » Thu, May 28, 2015, 11:50 PM UTC

devindotcom wrote: post #67

Kiro wrote: post #52

Earlier quoted context omitted.

You sound like someone who hasn't tried Oculus Rift. In that case I'm happy to inform you that you're wrong.

I've tried it up until Crescent Bay, and it's fabulous. But I still feel it is very much for carefully fashioned, virtual experiences, and will have trouble replicating things via video, however many gopros you put on the task.

"These mobile phones will never reach a mass market! The batteries are far too big!"

Re: Jump – Experiences like you're actually there

#123

Post by hyperion2010 » Thu, May 28, 2015, 11:52 PM UTC

harigov wrote: post #115

If they can do it in real-time, this can replace the LIDAR used in self-driving cars, which can reduce the cost and make them affordable.

LIDAR will almost always be a better option when you don't have particulate matter that obstructs the beam because you can measure the distances directly which vastly reduces latency. Round trip time to a remote server means you have already lost and also introduced some very nasty failure modes. This is the reason why valve uses a LIDAR like system for Lighthouse rather than try to do image processing like on the kinect.

Re: Jump – Experiences like you're actually there

#124

Post by drcross » Thu, May 28, 2015, 11:53 PM UTC

dimino wrote: post #71

mod wrote: post #68

Earlier quoted context omitted.

16 gopros isn't much more than a mid-range Canon DSLR, which TONS of amateurs manage to pony up the cash for. I don't think price will be a big inhibitor here, honestly.

Price is a huge inhibitor for a mid-range Canon DSLR, so it'll probably be a huge inhibitor for this system too.

Yeah but compare it to a Red or a Hasselblad and it's utter peanuts.

Re: Jump – Experiences like you're actually there

#125

Post by ChuckMcM » Thu, May 28, 2015, 11:54 PM UTC

jrowley wrote: post #118

josu wrote: post #108

Earlier quoted context omitted.

> get stereo from mono data Why do you call it mono data? Given how close are each lense from another, I'm guessing, that anything far enough (0.5m?) from the cameras is beign recorded by at least two cameras at all times.

Because simply stitching it will create one image/video stream from one perspective. If you have 2 eyes, you have stereo vision so can perceive depth. This technique allows for the synthesis of stereo video from a rig that doesn't directly capture stereo video. Pretty cool stuff

The point the GP was making was that you can treat each pair of cameras as a pair of 'eyes' for the stereoscopic analysis. And you get every combination of adjacent cameras as a pair so a ring of 8 cameras represents 8 distinct pairs (call them camera 'a' and camera 'a' + 1 clockwise)

By processing those streams you should be able to derive high quality depth information from the combined video streams, and given the overlap in views re-compute a 'view' from any direction.

A lot of early work was done using the streetview cameras as well to do street facing feature extraction using similar techniques but as those cameras were essentially single shot (discrete pictures over time rather than video) you end up with the "slewing" effect of moving from one spot to the next.

Presumably Google will go out and do street view with this sort of setup and get a continuous terrain option so that you can really walk around as if you were there (caveat occlusion effects)

Re: Jump – Experiences like you're actually there

#126

Post by drcross » Thu, May 28, 2015, 11:56 PM UTC

hyperion2010 wrote: post #123

harigov wrote: post #115

If they can do it in real-time, this can replace the LIDAR used in self-driving cars, which can reduce the cost and make them affordable.

LIDAR will almost always be a better option when you don't have particulate matter that obstructs the beam because you can measure the distances directly which vastly reduces latency. Round trip time to a remote server means you have already lost and also introduced some very nasty failure modes. This is the reason why valve uses a LIDAR like system for Lighthouse rather than try to do image processing like on the ki…

Imagine with high enough bandwidth where you could outsource the cost of a taxi to remote operators like drone pilots.

Re: Jump – Experiences like you're actually there

#127

Post by aggie » Thu, May 28, 2015, 11:59 PM UTC

wanderingstan wrote: post #44

untog wrote: post #29

Looks great, though as other commenters have pointed out, 16 GoPros makes for an expensive rig. I've been watching this space for a while and pretty much all of the consumer varieties (usually Kickstarter-backed) have pretty terrible video quality, so it looks like I'll be sitting this one out for a while yet.

They don't claim this to be a consumer product. As others have said, the cost of this is peanuts compared to professional cameras. I'd bet one of the first killer-apps for VR will be sports: Imagine several of these rigs set up around (and over) a football field, so that spectators can choose to "be" at the right point on the field to observe the game, and have complete freedom to look around the field.

I think you'd be surprised how nice it is to have a team of producers and camera operators doing the work of finding the interesting views for you. People attend live sports for the atmosphere and shared experience, not the view.

Re: Jump – Experiences like you're actually there

#128

Post by drcross » Fri, May 29, 2015, 12:01 AM UTC

jonknee wrote: post #33

swalsh wrote: post #12

So, if you're at an event or something (like sxsw) do you wear this as a hat? or have a large pole... or how does filming with this actually work?

Mounted on a tripod like any other camera system. Most people will view the content, few will record (16 cameras is a lot of money/hassle/data).

I guess you havnt used a GearVR yet then.

Re: Jump – Experiences like you're actually there

#129

Post by saluk » Fri, May 29, 2015, 12:06 AM UTC

bd wrote: post #60

Something that's maybe not clear from that page, but was mentioned in the Google IO keynote: this is not just stitching of multiple video streams. Google does some heavy-duty machine learning / computer vision on their servers to extract 3D information from these video streams (they mentioned having depth data). Then they presumably re-generate seamless stereoscopic 360-degree video from this 3D model. That's how the…

I had thought we would never get 360 + stereo. I wonder how much touch up work this will require and what kinds of scenes will provide problems with incorrect depth heuristics.

Re: Jump – Experiences like you're actually there

#130

Post by saluk » Fri, May 29, 2015, 12:10 AM UTC

GrantS wrote: post #73

DINKDINK wrote: post #54

Earlier quoted context omitted.

In order to do Stereoscopy properly, you need the axes of the camera barrel to intersect having a set of cameras all radially pointed outward doesn't achieve this.

The end user never sees the raw video in this case so that's not a requirement -- they're generating lots of synthetic stereoscopic views for the user by post-processing the video and accounting for depth. I'll be interested to see what data they're sending to the video player to provide both stereoscopy and 360-panoramic video at the same time. The methods I can think of offhand require real-time pixel-shuffling to…

I'm guessing it's planes all the way down.