Augmented Reality 3d Video on iPad with Kinect

#1

Post by iqram » Sat, Jul 02, 2011, 12:54 PM UTC

This is an awesome way to demonstrate what can be done with the Kinnect and the iPad String Augmented Reality SDK

Augmented Reality 3d Video on iPad with Kinect
youtube.com

Re: Augmented Reality 3d Video on iPad with Kinect

#2

Post by rkalla » Sat, Jul 02, 2011, 4:58 PM UTC

So damn clever... releasing an SDK for the Kinect[1] was one of the smartest things Microsoft could have ever done and I think has gone a long way to contribute to it being the fastest selling piece of hardware in history[2] (did anyone know that? I sure didn't, and barely believed it reading it).

This reminds me a lot of what Johnny Lee did with the Wii soon after it was released with head-tracking 3D[3]

Part of me wants to stop using a computer and take up beet-farming because the cool factor seems so much less than what these guys are doing.

Anyone on HN actively toying with a Kinect and want to share some video?

[1] http://research.microsoft.com/en-us/um/redmond/projects/kine...

[2] http://www.vgchartz.com/article/83375/kinect-is-the-fastest-...

[3] http://www.youtube.com/watch?v=Jd3-eiid-Uw

Re: Augmented Reality 3d Video on iPad with Kinect

#3

Post by chopsueyar » Sat, Jul 02, 2011, 5:34 PM UTC

Isn't the new Nintendo tablet doing something like this?

Also, why wasn't the piece of paper on the table showing up, only the books?

Re: Augmented Reality 3d Video on iPad with Kinect

#4

Post by dstein » Sat, Jul 02, 2011, 5:43 PM UTC

chopsueyar wrote: post #3

Isn't the new Nintendo tablet doing something like this? Also, why wasn't the piece of paper on the table showing up, only the books?

Ya it looks like the "augmented reality" part was faked. The angle of the video never quite matched up with where he was holding it either. In any case it's still a nice proof of concept.

Re: Augmented Reality 3d Video on iPad with Kinect

#5

Post by KingOfB » Sat, Jul 02, 2011, 5:58 PM UTC

Here's the launch video for String SDK. Looks pretty impressive, much smoother than many other implementations I've seen

http://www.poweredbystring.com/official-launch

Great demo video about 1:30 in to the launch video.

Re: Augmented Reality 3d Video on iPad with Kinect

#6

Post by chopsueyar » Sat, Jul 02, 2011, 5:59 PM UTC

dstein wrote: post #4

chopsueyar wrote: post #3

Isn't the new Nintendo tablet doing something like this? Also, why wasn't the piece of paper on the table showing up, only the books?

Ya it looks like the "augmented reality" part was faked. The angle of the video never quite matched up with where he was holding it either. In any case it's still a nice proof of concept.

I found the Nintendi E3 Wii U Demo.

It gets interesting around the 2:40 mark (the whole thing is pretty impressive)...

http://www.youtube.com/watch?v=DowHrWluHxg

Re: Augmented Reality 3d Video on iPad with Kinect

#7

Post by hammerdr » Sat, Jul 02, 2011, 6:08 PM UTC

chopsueyar wrote: post #3

Isn't the new Nintendo tablet doing something like this? Also, why wasn't the piece of paper on the table showing up, only the books?

I believe that in augmented reality programs you need a target. The piece of paper is that target. It allows for the AR program to know where to place the video and also how to rotate the camera as he is walking around to the side.

Re: Augmented Reality 3d Video on iPad with Kinect

#8

Post by hammerdr » Sat, Jul 02, 2011, 6:11 PM UTC

rkalla wrote: post #2

So damn clever... releasing an SDK for the Kinect[1] was one of the smartest things Microsoft could have ever done and I think has gone a long way to contribute to it being the fastest selling piece of hardware in history[2] (did anyone know that? I sure didn't, and barely believed it reading it). This reminds me a lot of what Johnny Lee did with the Wii soon after it was released with head-tracking 3D[3] Part of me…

I have no video, but I was able to get a WebGL-powered game world running that was controlled through a Kinect using DepthJS. This was done in just over a day of playing around. Was pretty cool when it worked.

The source code is available on github at https://github.com/thoughtworks/kinect-spike

Edit: My co-workers were looking at me with concerned looks that day. I would stand up and wave my arms for 30 seconds and then sit back down at random times throughout the day. :)

Re: Augmented Reality 3d Video on iPad with Kinect

#9

Post by dlib » Sat, Jul 02, 2011, 6:15 PM UTC

chopsueyar wrote: post #3

Isn't the new Nintendo tablet doing something like this? Also, why wasn't the piece of paper on the table showing up, only the books?

The piece of paper tells the app the position of the animation and is removed by the app from the video in the iPad. If you look closely you'll see the app projects some kind of wooden surface where the piece of paper was, this could be a pattern recognition thing (looking at the surrounding pattern and filling in the paper with that pattern) or just a custom job.

Re: Augmented Reality 3d Video on iPad with Kinect

#10

Post by rkalla » Sat, Jul 02, 2011, 6:50 PM UTC

hammerdr wrote: post #8

rkalla wrote: post #2

So damn clever... releasing an SDK for the Kinect[1] was one of the smartest things Microsoft could have ever done and I think has gone a long way to contribute to it being the fastest selling piece of hardware in history[2] (did anyone know that? I sure didn't, and barely believed it reading it). This reminds me a lot of what Johnny Lee did with the Wii soon after it was released with head-tracking 3D[3] Part of me…

I have no video, but I was able to get a WebGL-powered game world running that was controlled through a Kinect using DepthJS. This was done in just over a day of playing around. Was pretty cool when it worked. The source code is available on github at https://github.com/thoughtworks/kinect-spike Edit: My co-workers were looking at me with concerned looks that day. I would stand up and wave my arms for 30 seconds and…

Very impressive Derek -- how easy was it to work with the Kinect data via the API? Is the information coming out of the sensors pretty straight forward or does it just come in as a video feed with additional metadata (positional?)