Thanks for this! I have been working on something similar for an upcoming education app, record a course and play in the same app with a compact file format (1MB per minute, could be much less with some tricks). You can see a demo here : https://youtu.be/zcHAzQXm3Hg and more at https://explayn.me I will definitely check your lib, and will be happy to switch if it’s better !
Show HN: A tool for motion-capturing 3D characters using a VR headset
11–20 of 32 posts
Re: Show HN: A tool for motion-capturing 3D characters using a VR headset
#12This is great, well done, and thanks for open sourcing it. I started to build something like this early last year, but got too busy with another pet project and having a baby. Here's some demos and notes if you're interested. https://twitter.com/convey_it/status/1433163282597171200
Holy smoke that's so cool, thanks for sharing it! I love that your work helped increase the number of morphTargets in Three.js. That's real impact!
Re: Show HN: A tool for motion-capturing 3D characters using a VR headset
#13With an expensive Quest Pro you could capture your facial expressions using the headset's eye and face-tracking features.
Any more on this?
As for hands - is it able to track hands and fingers only or also an object within a hand? I can see this being useful to 'puppeteer' a 3D object by hand used as a proxy of it... Or a simple object within a hand.
Outside of maybe a cliff-climbing game I struggled with finding a reason for buying a headset at all. This seems like a plausible use for animation.
Re: Show HN: A tool for motion-capturing 3D characters using a VR headset
#14Thanks for this! I have been working on something similar for an upcoming education app, record a course and play in the same app with a compact file format (1MB per minute, could be much less with some tricks). You can see a demo here : https://youtu.be/zcHAzQXm3Hg and more at https://explayn.me I will definitely check your lib, and will be happy to switch if it’s better !
Wow that's super cool! That's definitely the biggest challenge - how to keep the size of the mocap data small.
Our library is not the best at that, since it simply records everything to Alembic files, which are quite heavy.
Best of luck with Explayn. I really like the idea of learning from mocapped lessons.
Re: Show HN: A tool for motion-capturing 3D characters using a VR headset
#15How well does it do with occlusion? Holding, or manipulating an object, and capturing those interactions.
Was wondering the same thing - if it does well there, it could be a really cheap way to, for example, motion-capture realistic weapon holding in an FPS, provided it can be extended to capture the arms as well using CV.
It would need some cleanup, but definitely usable.
Re: Show HN: A tool for motion-capturing 3D characters using a VR headset
#16This is great, well done, and thanks for open sourcing it. I started to build something like this early last year, but got too busy with another pet project and having a baby. Here's some demos and notes if you're interested. https://twitter.com/convey_it/status/1433163282597171200
Holy smoke that's so cool, thanks for sharing it! I love that your work helped increase the number of morphTargets in Three.js. That's real impact!
Thanks! Yeah, that felt good.
Re: Show HN: A tool for motion-capturing 3D characters using a VR headset
#17Thanks for this! I have been working on something similar for an upcoming education app, record a course and play in the same app with a compact file format (1MB per minute, could be much less with some tricks). You can see a demo here : https://youtu.be/zcHAzQXm3Hg and more at https://explayn.me I will definitely check your lib, and will be happy to switch if it’s better !
Wow that's super cool! That's definitely the biggest challenge - how to keep the size of the mocap data small. Our library is not the best at that, since it simply records everything to Alembic files, which are quite heavy. Best of luck with Explayn. I really like the idea of learning from mocapped lessons.
Thanks for the kind words! Yes, file size for live action or shipping directly in an app is always an issue. I may actually contribute some code, my file format is no rocket science, just a bare set of floats with some metas, not even some diff encoding between frames, so quite easy to interpret between languages and platforms.
Re: Show HN: A tool for motion-capturing 3D characters using a VR headset
#18Earlier quoted context omitted.
It does amazingly well. Last May Meta released their Hand Tracking 2.0 update, and the quality of the hand tracking improved enormously. You can see a comparison of 1.0 and 2.0 here: https://www.youtube.com/watch?v=K-l-RXxYnzs&t=4s&ab_channel=...
What about arms? Or would it have to rely on some realtime IK solution to sort them out?
For the arms you could use Meta's new Movement SDK: https://developer.oculus.com/documentation/unity/move-overvi...
Although in my opinion the arms still look a bit janky. You can see a demo of a sample they included with the SDK here: https://youtu.be/IQt4wTdGK64?t=419
Re: Show HN: A tool for motion-capturing 3D characters using a VR headset
#19With an expensive Quest Pro you could capture your facial expressions using the headset's eye and face-tracking features. Any more on this? As for hands - is it able to track hands and fingers only or also an object within a hand? I can see this being useful to 'puppeteer' a 3D object by hand used as a proxy of it... Or a simple object within a hand. Outside of maybe a cliff-climbing game I struggled with finding a r…
Well consider this sample from the Meta Movement SDK that allows you to control the facial expressions of a character with the face tracking of the Quest Pro: https://youtu.be/IQt4wTdGK64?t=315
Our tool could record those facial expressions into an Alembic file.
And as for holding objects, our tool could capture those too. Meta's Interaction SDK is excellent. Here's another sample they provide: https://youtu.be/WS4vtcwm8Zw?t=1
And I have to say that the cliff-climbing game is really fun.
Re: Show HN: A tool for motion-capturing 3D characters using a VR headset
#20This could be great for people doing work with various sign languages.