Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On-Device, Real-Time Hand Tracking with MediaPipe

ai.googleblog.com

Post reply on HN
Print view View original

61–70 of 70 posts

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#61

Post by Mathnerd314 »

dekhn wrote: post #59
Mathnerd314 wrote: post #29

Earlier quoted context omitted.

Probably because it uses TensorFlow (lite) and there aren't many (any?) VPLs supporting C++ integration. And also Google has NIH syndrome, it's hard to think of any outside projects they use besides the Linux software stack and LLVM.

Google depends on thousands of external software projects. They heavily support a VPL, Scratch.

AFAICT the relationship for Scratch is that Google wrote a new VPL (Blockly) and the MIT Media Lab released it as Scratch 3.0.

I guess the "thousands" is from https://opensource.google.com/? In a few minutes of browsing, I couldn't find anything besides Bullet that wasn't just Google releasing something as open source.

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#62

Post by undoware »

debrice wrote: post #60
undoware wrote: post #40

Earlier quoted context omitted.

Try drawing hands. (If you're a non-artist like myself.) It's impressively hard --- they are complex artefacts we can't quite see clearly because we are so used to them. There are a lot of, as it were, polygons. If, after staring at these things for several decades, I still can't draw them with my eyes closed, I will assume that an AI would not find it easy to think about them either.

I think it’s the opposite problem. I can see an AI being able to very easily draw what it sees but struggle at interpreting it. When you learn to draw, one of the first step is relearning to see without interpretation. A better example is example is how kids draw stick figures so easily and how hard it is for computer to do so: for decades we had to put reflector interpreted as dots used to render vector... use to re…

What a great point, thanks!

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#63

Post by joshspankit »

gfodor wrote: post #37
joshspankit wrote: post #28

Earlier quoted context omitted.

The LeapMotion actually use ultrasonics for measurements and then guesses to translate that to hand-tracking. Doing it fully from a camera may actually improve on it if done right.

That’s the magic leap. The original leap motion uses stereoscopic infrared cameras afaik.

Nope, you’re totally right. Don’t know where I got the idea that the little Leap Motion was ultrasonic. Apologies to the readers.

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#64

Post by h2odragon »

hirundo wrote: post #33

> We are excited to see what you can build with it! The killer app is typing. Qwerty would be nice for a transition, but someone please invent a gesture "keyboard" more optimal for a free floating hand. Because of the lack of feedback I imagine it couldn't be as good as an actual keyboard. But it could be brilliant as an away from keyboard keyboard.

There's still ergonomics. Say we teach computers to recognize casual ASL (which is a big job for people learning ASL, but whatever)... You're not going to be able to spend the time using that input method you could using a keyboard, because of simple fatigue.

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#65

Post by cromwellian »

eitland wrote: post #51
cromwellian wrote: post #50

Earlier quoted context omitted.

It’s an open source library, so you can’t “shut it down”, it’s a gift to the community and people can continue to use it if Google doesn’t. Honestly these kinds of comments are getting silly and applied out of context, it’s almost like there needs to be. Godwin’s law for dredging up Reader’s cancellation or conspiracy theories about how finger tracking is going to be used for ad recommendation.

Edit: upvoted for the point that it is open source. I never cared about Reader. For me it was 5 or 10 or even more different other things like - Killing xmpp through extend, embrace extinguish - Creating Google+ without making it open - Killing Google+ - ReCaptcha v2 and v3 - Constantly pushing the boundaries for tracking, now also buying credit card logs - etc etc

Google did not kill XMPP, the number of people who actually used Google Chat client was tiny. What killed XMPP was ICQ, MSN, Yahoo, Facebook, and text messaging.

Your comments on federation are mistaken IIRC. Google Chat supported federation from 2005 to 2014, it was dropped in a Google I/O announcement because none of the other players mentioned above who were gobbling up the messaging market reciprocated, they instead took advantage of Googles support of federation to onboard users to their proprietary networks. Google Chats user base was declining and the network effort of the proprietary social networks was pulling everyone in.

A similar thing happened with OpenID. Google initially supported it, and Social networks used it for new user signup account creation, but did not allow it for signin.

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#66

Post by eitland »

cromwellian wrote: post #65
eitland wrote: post #51

Earlier quoted context omitted.

Edit: upvoted for the point that it is open source. I never cared about Reader. For me it was 5 or 10 or even more different other things like - Killing xmpp through extend, embrace extinguish - Creating Google+ without making it open - Killing Google+ - ReCaptcha v2 and v3 - Constantly pushing the boundaries for tracking, now also buying credit card logs - etc etc

Google did not kill XMPP, the number of people who actually used Google Chat client was tiny. What killed XMPP was ICQ, MSN, Yahoo, Facebook, and text messaging. Your comments on federation are mistaken IIRC. Google Chat supported federation from 2005 to 2014, it was dropped in a Google I/O announcement because none of the other players mentioned above who were gobbling up the messaging market reciprocated, they inst…

Thanks facts always appreciated, you might very well be right about those two. I still think Google has earned their current reputation.

(And don't for a second think I think less of Google than the rest, it is probably just that I used to trust Google somewhat more than the rest.)

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#67

Post by andrerm »

kjar wrote: post #46
bigiain wrote: post #42

Earlier quoted context omitted.

> A combination voice recognition + gesture recognition + eye tracking would make ... retail surveillance much more effective and privacy eroding...

if they do it all on device and do not transmit out, then the privacy concern seems squelched. On retail tracking yeah it's messed, practically no one knows their movements are tracked on their phones - cell tower triangulation, GPS, then the kicker NFC. I love / hate tech. It's powerful stuff. [edit] - the cell tower triangulation is forever given the nature of the tech.

> all on device and do not transmit out

Like Apple? Retorical question. My point is: this stays in device is just propaganda. It can be done but will never happen

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#68

Post by cromwellian »

eitland wrote: post #66
cromwellian wrote: post #65

Earlier quoted context omitted.

Google did not kill XMPP, the number of people who actually used Google Chat client was tiny. What killed XMPP was ICQ, MSN, Yahoo, Facebook, and text messaging. Your comments on federation are mistaken IIRC. Google Chat supported federation from 2005 to 2014, it was dropped in a Google I/O announcement because none of the other players mentioned above who were gobbling up the messaging market reciprocated, they inst…

Thanks facts always appreciated, you might very well be right about those two. I still think Google has earned their current reputation. (And don't for a second think I think less of Google than the rest, it is probably just that I used to trust Google somewhat more than the rest.)

I work for Google, I definitely think the behavior of Apple, Microsoft, Yahoo, Facebook, and others hurt the idea that "open wins". Google tried hard with XMPP, OpenID, OAuth, ActivityStreams, WebFinger, OpenSearch, FOAF, OpenSocial, et al to build distributed, federated solutions. Even Google Wave was designed with federation support. At some point, even they had to admit, they were losing.

The problem is, tech geeks love it (open), but consumers don't care, and after the smartphone revolution, it's far easier to build siloed apps, and get people into things like iMessage, WeChat, or WhatsApp, than it is to get people to adopt a federated protocol.

If Email had been invented in the modern era with consumers controlling what wins, we would not have SMTP. We'd have proprietary platform specific mailboxes, and people would have to create accounts to send people mail on a platform.

The internet had a brilliant run in the 80s, and early 90s, before the great masses arrived, back when protocols were designed by people interested in technical capabilities, not money, when these things were hashed out at IETF meetups and mailing lists.

Consumer behavior and investment decisions today inherently force centralization I think, and it's hard to build a truly open system these days.

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#69

Post by soylentgraham »

jedimastert wrote: post #3

I've started to get really interested in lower-budget and/or easier motion tracking for special effects. I've been looking at optical motion tracking with a multi camera setup, and optical facial tracking. With the right math and assumptions, you could capture a full performance with little to no specialized equipment. I've been wondering if ML could output enough detail to make it feasible.

With the right models, and gluing it all together yeah.

For some commercials we've dropped xsens suits for openpose. Facial capture from afar needs too much in the way of exaggerated movement, but for mouth capture, audio processing gave more pleasing results. 3D models still aren't here but for cameras that are staying basically in one plane it's good.

Used open pose initially to correct capture suit drift live, but with some math (I'm a game & computer vision dev) translating to 3D was pretty good. As always, you just have to fix outliers.

Re: On-Device, Real-Time Hand Tracking with MediaPipe

#70

Post by debrice »

visarga wrote: post #39
debrice wrote: post #30

As a engineer I can see how truly amazing feat that is. As a human being I'm staggered that it took so much effort into AI and machine learning to do so little.

> so little Nature took 85 million years to perfect the hand, and dexterous use takes 1-2 years of training for babies. Interpreting the hands of other takes longer.

That's very good point. I would correct one thing, a baby can learn sign language (aka interpret other's hands gesture) by 4 to 6 months old.
Post reply on HN