Live data from Hacker News

On-Device, Real-Time Hand Tracking with MediaPipe

ai.googleblog.com

21–30 of 70 posts

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#21
post #17

Thinking of all the baseball applications here: catcher signals, third base coach, head coach etc.

Why would it be useful to have technology detect signals? Aren't people already going to be doing it?

Genuine question, I don't know how baseball works.

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#24
post #5

Wasn't this posted here about 2 days ago?

Might have been, sorry - came across this via a fresh story on Gamasutra this evening, just realised the original blog post is from last week.

FWIW, thank you for (re)posting. This is very intriguing and I did not know about it.

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#26
post #14

Earlier quoted context omitted.

You can buy a LeapMotion and stick it to the front of Rift right now. I've done it, it's amazingly immersive and feels fantastic.

I really wonder how this compares to the LeapMotion tracking. My suspicion is that the leap tracking is now hardened by years of real world experience, so it's probably ahead of anything that's still in the R&D stage. But hard to know without testing it.

This article doesn't actual mention anywhere but it implies it's doing all this with a regular camera. LeapMotion and others use more complex sensors. The ML approach is really impressive but getting clearer input would seem to be a more reliable approach.

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#27

I've started to get really interested in lower-budget and/or easier motion tracking for special effects. I've been looking at optical motion tracking with a multi camera setup, and optical facial tracking. With the right math and assumptions, you could capture a full performance with little to no specialized equipment. I've been wondering if ML could output enough detail to make it feasible.

I’m always keeping an eye on tracking results to see if they’re doing ML (or nother technique) to extrapolate the muscle movements so that those movements can be translated directly in to a character. Especially watching this in the fascial tracking since these “skin-stretch” animations are so jarring.

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#28
post #26
post #14

Earlier quoted context omitted.

I really wonder how this compares to the LeapMotion tracking. My suspicion is that the leap tracking is now hardened by years of real world experience, so it's probably ahead of anything that's still in the R&D stage. But hard to know without testing it.

This article doesn't actual mention anywhere but it implies it's doing all this with a regular camera. LeapMotion and others use more complex sensors. The ML approach is really impressive but getting clearer input would seem to be a more reliable approach.

The LeapMotion actually use ultrasonics for measurements and then guesses to translate that to hand-tracking. Doing it fully from a camera may actually improve on it if done right.

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#29
post #11

The underlying project MediaPipe looks pretty cool: https://github.com/google/mediapipe/blob/master/mediapipe/do... I wonder why they didn't build it on top of one of the gazillion flow-based visual programming languages instead?

Probably because it uses TensorFlow (lite) and there aren't many (any?) VPLs supporting C++ integration.

And also Google has NIH syndrome, it's hard to think of any outside projects they use besides the Linux software stack and LLVM.

Post reply on HN