Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: A tool for motion-capturing 3D characters using a VR headset

diegomacario.github.io

Post reply on HN
Print view View original

1–10 of 32 posts

Show HN: A tool for motion-capturing 3D characters using a VR headset

#1

Post by diegomacario »

Hi everyone! I'm one of the authors of this project. The demo you see here is powered by a tool that I recently helped develop and open-source at Shopify called handy. You can find the repo here: https://github.com/Shopify/handy

Most people don't realize that VR headsets have become really capable motion capture platforms, so we decided to release this tool to bring motion capture into the hands of everyone who owns a headset.

With a cheap Quest 2 you can capture your hands using the headset's hand-tracking feature and your head. With an expensive Quest Pro you could capture your facial expressions using the headset's eye and face-tracking features.

Thanks for checking this project out! I'm here to answer questions if you have any.

Show HN: A tool for motion-capturing 3D characters using a VR headset
diegomacario.github.io

Re: Show HN: A tool for motion-capturing 3D characters using a VR headset

#4

Post by karim79 »

klaussilveira wrote: post #2

How well does it do with occlusion? Holding, or manipulating an object, and capturing those interactions.

Was wondering the same thing - if it does well there, it could be a really cheap way to, for example, motion-capture realistic weapon holding in an FPS, provided it can be extended to capture the arms as well using CV.

Re: Show HN: A tool for motion-capturing 3D characters using a VR headset

#5

Post by diegomacario »

ngokevin wrote: post #3

Plugging in this one we did for reference a long time ago! Capture + share dances https://blog.mozvr.com/a-saturday-night/

I love this so much, thanks for sharing it! We have been thinking about similar concepts at Shopify. Mocapped performances convey so much more emotion than images or videos.

Re: Show HN: A tool for motion-capturing 3D characters using a VR headset

#6

Post by diegomacario »

klaussilveira wrote: post #2

How well does it do with occlusion? Holding, or manipulating an object, and capturing those interactions.

It does amazingly well. Last May Meta released their Hand Tracking 2.0 update, and the quality of the hand tracking improved enormously. You can see a comparison of 1.0 and 2.0 here: https://www.youtube.com/watch?v=K-l-RXxYnzs&t=4s&ab_channel=...

Re: Show HN: A tool for motion-capturing 3D characters using a VR headset

#8

Post by endergen »

This is great, well done, and thanks for open sourcing it.

I started to build something like this early last year, but got too busy with another pet project and having a baby. Here's some demos and notes if you're interested. https://twitter.com/convey_it/status/1433163282597171200

Re: Show HN: A tool for motion-capturing 3D characters using a VR headset

#9

Post by diegomacario »

daniel_iversen wrote: post #7

That’s cool! What would the typical use case for this be (and why did Shopify build it)?

We actually built this to easily add hands to this concept video: https://twitter.com/StrangeNative/status/1613218237969494017...

Animating hands by hand (no pun intended) is horrendously difficult, so our 3D artist asked if we could help.

We built this within Shopify's Spatial Commerce team, which is a team that explores the intersection between AR/VR and commerce. Here are some of our other projects:

https://twitter.com/StrangeNative/status/1544666221181734912... https://twitter.com/StrangeNative/status/1562450155080597505...

Re: Show HN: A tool for motion-capturing 3D characters using a VR headset

#10

Post by klaussilveira »

diegomacario wrote: post #6
klaussilveira wrote: post #2

How well does it do with occlusion? Holding, or manipulating an object, and capturing those interactions.

It does amazingly well. Last May Meta released their Hand Tracking 2.0 update, and the quality of the hand tracking improved enormously. You can see a comparison of 1.0 and 2.0 here: https://www.youtube.com/watch?v=K-l-RXxYnzs&t=4s&ab_channel=...

What about arms? Or would it have to rely on some realtime IK solution to sort them out?
Post reply on HN