Show HN: Controlling 3D models with voice and hand gestures

#1

Post by getToTheChopin » Tue, Jun 03, 2025, 2:45 PM UTC

I'm sharing my project to control 3D models with voice commands and hand gestures:

- use voice commands to change interaction mode (drag, rotate, scale, animate)

- use hand gestures to control the 3D model

- drag/drop to import other models (only GLTF format supported for now)

Created using threejs, mediapipe, web speech API, rosebud AI, and Quaternius 3D models

Githhub repo: https://github.com/collidingScopes/3d-model-playground

Demo: https://xcancel.com/measure_plan/status/1929900748235550912

I'd love to get your feedback! Thank you

Show HN: Controlling 3D models with voice and hand gestures
github.com

Re: Show HN: Controlling 3D models with voice and hand gestures

#2

Post by getToTheChopin » Tue, Jun 03, 2025, 3:09 PM UTC

Here's a quick video demo showing how it works: https://x.com/measure_plan/status/1929900748235550912

Re: Show HN: Controlling 3D models with voice and hand gestures

#3

Post by xnx » Tue, Jun 03, 2025, 3:12 PM UTC

Sounds very cool, but I could not make sense of the on-screen instructions. Some images or animations would go a long way to explain the controls.

Re: Show HN: Controlling 3D models with voice and hand gestures

#4

Post by ednite » Tue, Jun 03, 2025, 3:12 PM UTC

Great job! Looks very useful for interactive content creations and product showcasing. Definitely will testing it more. Thanks for sharing.

Re: Show HN: Controlling 3D models with voice and hand gestures

#5

Post by ActionHank » Tue, Jun 03, 2025, 3:13 PM UTC

getToTheChopin wrote: post #2

Here's a quick video demo showing how it works: https://x.com/measure_plan/status/1929900748235550912

[flagged]

Re: Show HN: Controlling 3D models with voice and hand gestures

#6

Post by getToTheChopin » Tue, Jun 03, 2025, 3:15 PM UTC

xnx wrote: post #3

Sounds very cool, but I could not make sense of the on-screen instructions. Some images or animations would go a long way to explain the controls.

Sorry about that, the instructions need to be improved.

Does this video demo help?

https://x.com/measure_plan/status/1929900748235550912

If it makes it clearer, I'll upload it to the github repo directly

Re: Show HN: Controlling 3D models with voice and hand gestures

#7

Post by [deleted] » Tue, Jun 03, 2025, 3:22 PM UTC

ActionHank wrote: post #5

getToTheChopin wrote: post #2

Here's a quick video demo showing how it works: https://x.com/measure_plan/status/1929900748235550912

[flagged]

[deleted]

Re: Show HN: Controlling 3D models with voice and hand gestures

#8

Post by pvg » Tue, Jun 03, 2025, 3:24 PM UTC

getToTheChopin wrote: post #2

Here's a quick video demo showing how it works: https://x.com/measure_plan/status/1929900748235550912

https://xcancel.com/measure_plan/status/1929900748235550912 for the tweetless

Re: Show HN: Controlling 3D models with voice and hand gestures

#9

Post by getToTheChopin » Tue, Jun 03, 2025, 3:30 PM UTC

ednite wrote: post #4

Great job! Looks very useful for interactive content creations and product showcasing. Definitely will testing it more. Thanks for sharing.

yes I'd love to go further with this concept so that 3D / CAD designers could easily present their models during video calls.

thank you!

Re: Show HN: Controlling 3D models with voice and hand gestures

#10

Post by appendixv2 » Tue, Jun 03, 2025, 4:07 PM UTC

[flagged]