Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: YoHa – A practical hand tracking engine

handtracking.io

Post reply on HN
Print view View original

21–30 of 67 posts

Re: Show HN: YoHa – A practical hand tracking engine

#22

Post by b-3-n »

phailhaus wrote: post #14

An "undo" gesture seems necessary, it was a bit too easy to accidentally wipe the screen. Aside from that, this is fantastic! Love to see what WASM is enabling these days on the web.

Thank you for the feedback. Indeed such a functionality would be nice. One could solve this via another hand pose or in some way also with the existing hand poses. E.g. make a fist for say 2 seconds to clear the whole screen. Anything shorter will just issue an "undo".

YoHa uses tfjs.js which provides several backends for computation. One indeed uses WASM, the other one is WebGL based. The latter one is usually the more powerful one.

Re: Show HN: YoHa – A practical hand tracking engine

#23

Post by itake »

pantulis wrote: post #16
itake wrote: post #11

I think these tools are super interesting, but I tools like this marginalize users with non-standard number of limbs or fingers.

This is a very valid point, but as a counter argument the technique implemented here could be adapted to help users with other needs like say, a browser extension that can help you navigate back and forward with the blink of an eye.

This all gets complicated, because not everyone has 2 eyes :-/.

You end up with complicated systems trying to cover all of the edge cases.

Re: Show HN: YoHa – A practical hand tracking engine

#25

Post by b-3-n »

karxxm wrote: post #21

Would you provide the related paper to this approach?

In contrast to similar works there is no dedicated paper that presents e.g. the neural network or the training procedure. Of course ideas from many papers influenced this work and I can't list them all here. Maybe it helps that the backbone of the network is very similar to MobileNetV2 (https://arxiv.org/abs/1801.04381). Let me know if you have any more questions in that regard.

Re: Show HN: YoHa – A practical hand tracking engine

#27

Post by rpmisms »

itake wrote: post #11

I think these tools are super interesting, but I tools like this marginalize users with non-standard number of limbs or fingers.

So does the real world. Things are hard to do with disabilities. That's what the word means. This has great potential, and it's not worth shutting down because some people aren't able to use it.

I can also see this being very helpful for people who have cerebral palsy, for example. Larger movements are easier, this might help someone use the web more easily.

Re: Show HN: YoHa – A practical hand tracking engine

#28

Post by b-3-n »

inetsee wrote: post #18

My first question is whether this has the capability of being adapted to interpret/translate American Sign Language (ASL)?

Thank you for this inspiring question. For interpreting sign language you need multi-hand support which YoHa is currently lacking. Apart from that you likely also need to account for the temporal dimension which YoHa also does not do right now. If those things were implemented I'm confident that it would produce meaningful results.

Re: Show HN: YoHa – A practical hand tracking engine

#29

Post by b-3-n »

borplk wrote: post #26

Very impressive. I want something like this so I can bind hand gestures to commands. For example scroll down on a page by a hand gesture.

One can build this pretty easily for a website that you are hosting with the existing API (https://github.com/handtracking-io/yoha/tree/master/docs).

However, you likely want this functionality on any website that you are visiting for which you probably need to build a browser extension. I haven't tried incorporating YoHa into a browser extension but if somebody were to try I'd be happy to help.

Post reply on HN