Live data from Hacker News

ML turns video of a 360° turn into 3D model of a person

sciencemag.org

21–30 of 33 posts

Re: ML turns video of a 360° turn into 3D model of a person

#21

How is this ML? They use a CNN for foreground segmentation, a minor step in their pipeline. But the major contribution seems to be putting the silhouettes in a common reference frame. I sincerely hope sciencemag isn’t putting ML in the title purely to jump on the bandwagon.

It's someone standing in front of a green screen. You don't need ML to find a person's silhouette.

Re: ML turns video of a 360° turn into 3D model of a person

#23
post #16

Oh how I hate marketing speech. First of all, the title should include "video of a predefined 360° turn". And then they say something along the lines of "average accuracy of about 5mm" for joining the constructed modeled joints to their model, while you see the body wobbling around happily. This is an impressive demo, but gah!

Ok, we'll give it a 360° turn above.

Re: ML turns video of a 360° turn into 3D model of a person

#24
post #17

As an artist, my first thought is I wonder what happens if you try giving this a series of drawings .

you'd probably need a lot of drawings, I wonder what's the sampling rate the thing uses it's a cool idea though :)

They say “standard” video is the source, so it would likely be on the order of 30 or 60 fps. Seems to be around a couple hundred frames, give or take, though I suspect it could get _something_ out of fewer frames, and more would just incrementally improve the model.

I would expect minor textural differences in a hand-drawn or painted source would make it a lot harder to correlate points between frames, but it’s an interesting idea to think about!

Re: ML turns video of a 360° turn into 3D model of a person

#25

How is this ML? They use a CNN for foreground segmentation, a minor step in their pipeline. But the major contribution seems to be putting the silhouettes in a common reference frame. I sincerely hope sciencemag isn’t putting ML in the title purely to jump on the bandwagon.

It's someone standing in front of a green screen. You don't need ML to find a person's silhouette.

To be fair, they do have examples that aren’t chroma keyed; they just lead with one that is.

Which is not to say that ML is necessary for this sort of computer vision task, but I wonder if it yields better or sharper results than other techniques?

Re: ML turns video of a 360° turn into 3D model of a person

#26

Earlier quoted context omitted.

Can you elaborate?

Makes forging facial biometrics easier.

In the case of Face ID, at least, you’d still have to transfer the measurements into the physical world, in a way that fools a system that has ostensibly been designed not to be fooled by masks.

Re: ML turns video of a 360° turn into 3D model of a person

#27

Earlier quoted context omitted.

Makes forging facial biometrics easier.

In the case of Face ID, at least, you’d still have to transfer the measurements into the physical world, in a way that fools a system that has ostensibly been designed not to be fooled by masks.

Like a 3D printed model?

Re: ML turns video of a 360° turn into 3D model of a person

#28

How is this ML? They use a CNN for foreground segmentation, a minor step in their pipeline. But the major contribution seems to be putting the silhouettes in a common reference frame. I sincerely hope sciencemag isn’t putting ML in the title purely to jump on the bandwagon.

What a farce. The use of the ML for background subtraction is almost inconsequential to the contribution of the paper and the result.

Re: ML turns video of a 360° turn into 3D model of a person

#30
post #19

I wonder if will see a future soon where a director can fully edit the positions and physical actions of the actors at post production. basically, the whole scenes will be transferred to believable 3d models seemlessly, and you can reanimate parts of everything. I feel like that's doing to happen for sure, for big Hollywood productions at least (like the Marvel stuff)

This already happens a lot, most VFX heavy productions will have digital doubles of the main cast, and they can be used for as simple a reason as reframing a shot.
Post reply on HN