It would be cool privacy-wise to have some sort of virtual webcam that could share such an animation rather than what the webcam actually sees.
As a commercial solution there's facerig+live2d that already does that.
Pose Animator: SVG animation tool using real-time TensorFlow.js models
51–60 of 78 posts
Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models
#52This is like creating a live version of author drawings, e.g. of New Yorker contributors: https://www.newyorker.com/contributors How interesting would it be to have your own live emotionally expressive avatar for videoconferencing, when you don't want to worry about your hair, makeup, lighting, or general visual state at all?
David Foster Wallace ruminated on this in Infinite Jest.
"The proposed solution to what the telecommunications industry's psychological consultants termed Video-Physiognmoic Dsyphoria (or VPD) was, of course, the advent of High-Definition Masking. Mask-wise, the initial option of High-Definition Photographic Imaging — i.e. taking the most flattering elements of a variety of flattering multi-angle photos of a given phone-consumer and‚ thanks to existing image-configuration equipment already pioneered by the cosmetics and law-enforcement industries — combining them into a wildly attractive high-def broadcastable composite of a face wearing an earnest, slightly overintense expression of complete attention."
Always thought it was fascinating that he came up with this in 1996!
Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models
#53This is awesome!
Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models
#54It would be cool privacy-wise to have some sort of virtual webcam that could share such an animation rather than what the webcam actually sees.
This is fairly easy to wire up on Linux using v4l2loopback and pyfakewebcam. I am currently using a little setup that uses OpenCV to acquire frames from the real camera, TensorFlow/BodyPix to compute an alpha mask for the foreground (me) and then OpenCV again to transform and composite myself behind news desks and into car infotainment screens and the like, eventually writing it to a virtual webcam I can use from Zoo…
Do you have any more details written up anywhere on how you do this? That sounds like a great project.
Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models
#55It would be cool privacy-wise to have some sort of virtual webcam that could share such an animation rather than what the webcam actually sees.
OBS studio can do that.
Please more details - I can not find any feature on the OBS website similar to what we see here. Do we need some plugin / extension? Please give us some urls, thank you!
Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models
#56Wow, this would change how animation is done completely! If there existed an easy way to create animated character cartoons. It would launch a thousand southpark/rick and morty type of shows. A team with 10k can launch a show.
The industry has had several tools like this for year’s. it can be useful but it hasn’t caught on in the past
please give us urls to other open source tools the industry uses for this kind of work, thank you very much.
Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models
#57This is like creating a live version of author drawings, e.g. of New Yorker contributors: https://www.newyorker.com/contributors How interesting would it be to have your own live emotionally expressive avatar for videoconferencing, when you don't want to worry about your hair, makeup, lighting, or general visual state at all?
"You can look like a gorilla or a dragon or a
giant talking penis in the Metaverse. Spend five minutes walking down the Street and you will see all of these.
Hiro's avatar just looks like Hiro, with the difference that no matter what Hiro is wearing in Reality, his avatar always wears a black leather kimono. Most hacker types don't go in for garish avatars, because they know that it takes a
lot more sophistication to render a realistic human face than a talking penis. Kind of the way people who really know clothing can appreciate the fine details that separate a cheap gray wool suit from an expensive hand-tailored gray wool
suit.", Neil Stephenson, Snowcrash, 1992.
Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models
#58It would be cool privacy-wise to have some sort of virtual webcam that could share such an animation rather than what the webcam actually sees.
This is a great idea! It should be great for bandwidth too - rather than sending full cam frames multiple times a second, it only needs to send deformation information describing movements.
Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models
#59This is great :) 10 years ago I'd been using Animata http://animata.kibu.hu + kinect, this rolls it all into one web based thing. cool stuff. Would it be easy to move it to a local GPU accelerated version for more FPS?
Re: Pose Animator: SVG animation tool using real-time TensorFlow.js models
#60Wow really cool! I'm wondering how much work it is to define the specific anchor/interest points on the SVG for the correct mapping to occur.
But I guess since "modern" illustrations are quite minimal, said work probably shouldn't take too long.