Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pose Animator: SVG animation tool using real-time TensorFlow.js models

github.com

Post reply on HN
Print view View original

61–70 of 78 posts

Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models

#61

Post by soylentgraham »

iseanstevens wrote: post #59

This is great :) 10 years ago I'd been using Animata http://animata.kibu.hu + kinect, this rolls it all into one web based thing. cool stuff. Would it be easy to move it to a local GPU accelerated version for more FPS?

Hmm tfjs does use the gpu, and there are plenty of small models for posenet to run fast on mobile. Maybe this is an old version. (Or maybe the bottleneck is in the face or svg stuff) But to answer your question, yeah it could be faster (everything could be faster given enough time! :)

Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models

#62

Post by soylentgraham »

GordonS wrote: post #58
p4bl0 wrote: post #24

It would be cool privacy-wise to have some sort of virtual webcam that could share such an animation rather than what the webcam actually sees.

This is a great idea! It should be great for bandwidth too - rather than sending full cam frames multiple times a second, it only needs to send deformation information describing movements.

Hah that's what video compressors do!

Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models

#63

Post by rw2 »

jjeaff wrote: post #49
rw2 wrote: post #48

Earlier quoted context omitted.

Good to know, it's a awesome feature I am surprised I don't see more successful shows made with this on reddit and youtube

That's because the animation is very much the least important part of a successful show. It's the writing and voice acting. South Park is a great example of this. It's successful because it is funny and edgy. The animation could have gone a lot of different ways and had similar success I think.

I think fundraising is the hardest part of a project, especially for creatives. If a group of writers can wrangle some amateur voice actors to work, (Which I think would be the lowest paid actors) it should be easy to get a project up.

Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models

#64

Post by viraptor »

crazygringo wrote: post #37

This is like creating a live version of author drawings, e.g. of New Yorker contributors: https://www.newyorker.com/contributors How interesting would it be to have your own live emotionally expressive avatar for videoconferencing, when you don't want to worry about your hair, makeup, lighting, or general visual state at all?

You can do that with some premade and custom models via https://facerig.com/ already. They're not perfect, but they aree pretty expressive.

Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models

#65

Post by aantix »

crazygringo wrote: post #37

This is like creating a live version of author drawings, e.g. of New Yorker contributors: https://www.newyorker.com/contributors How interesting would it be to have your own live emotionally expressive avatar for videoconferencing, when you don't want to worry about your hair, makeup, lighting, or general visual state at all?

IOS has the animated animal characters that mimic gestures pretty well.

Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models

#67

Post by eggy »

aantix wrote: post #65
crazygringo wrote: post #37

This is like creating a live version of author drawings, e.g. of New Yorker contributors: https://www.newyorker.com/contributors How interesting would it be to have your own live emotionally expressive avatar for videoconferencing, when you don't want to worry about your hair, makeup, lighting, or general visual state at all?

IOS has the animated animal characters that mimic gestures pretty well.

I was playing with OpenFrameworks and Kinect years ago, and you could pretty much do this, so I am curious why you would TF.js instead of simple OpenCV or other libraries that don't need to to use ML or DL. Or am I mistaken, and it is using simple bits of TS.js?

We had a large, flip-disc (or dot) wall at the site in Macau in 2012 that we purchased from a company that was a wall of black and white discs or dots that would flip to create a cool effect by tracking people in front of the wall with Kinect units in real time. It also made a cool clicky sound like old train/airport physical arrival/departure boards did.

I had a feature on my HTC phone or early Skype over 10 years ago where a cartoon cat mimicked my mouth, eyes, and head movement live on camera, which I can't find the reference to, but I have a screen recording of it when talking with my kids in the US from Macau.

I also remember using animata, a software that animated 2D puppet-like cut-outs to the music, I played with over 7 years or so ago, that was really cool [1 YouTube].

[1] https://www.youtube.com/watch?v=Dz8OMxB8m_M

Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models

#68

Post by Sephr »

p4bl0 wrote: post #24

It would be cool privacy-wise to have some sort of virtual webcam that could share such an animation rather than what the webcam actually sees.

The Snap Camera desktop app offers features somewhat similar to what you're seeking, as long as you combine it with the right filters. I imagine that the open source repo in the OP will eventually be used in an app like this.

Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models

#69

Post by Budabellly »

ravila4 wrote: post #27

This is very cool! I wonder if using input from multiple cameras, plus a 3D rig can improve the accuracy of the rendering.

Certainly, there are lots of apps leveraging the TrueDepth sensor on iPhones that reach an even higher level of fidelity as far is pose estimation is concerned.

The limiting factor for accuracy in a lot of these technologies is the actual rigging process of the characters, probably because that is very difficult to standardize or generalize across different geometries, art styles, animation drivers, 2D vs 3D, etc.

Post reply on HN