11–20 of 24 posts
Really neat article. I have always wondered if these techniques are used at all in higher fps interpolations which always seem... off.
I hit upon a similar idea back in about 2000... the key at the time was video encoding hardware that was designed to incorporate MPEG compression. Pulling motion vectors from the encoded stream could have given realtime optical flow way back then. Unfortunately I was nowhere good enough as a programmer to take advantage of it for real projects. A team at MIT had a paper about this that I was able to find. Had a page…
https://picamera.readthedocs.io/en/release-1.13/recipes2.htm...
I hit upon a similar idea back in about 2000... the key at the time was video encoding hardware that was designed to incorporate MPEG compression. Pulling motion vectors from the encoded stream could have given realtime optical flow way back then. Unfortunately I was nowhere good enough as a programmer to take advantage of it for real projects. A team at MIT had a paper about this that I was able to find. Had a page…
Earlier quoted context omitted.
We're doing this with H.264 at work. Pretty useful since we need to generate the encoded stream anyway, so now we get motion estimation for free.
Awesome. Which encoder provides this? Can you pull it from cuvid?
We were originally looking at analyzing the h264 bitstream in software and getting the vectors out that way. Honestly I don't think that would be extraordinarily difficult, there's already code in ffmpeg to draw the motion vectors as arrows on the decoded frame, so that would be a good place to look if there aren't any preexisting apis or tools to extract them.
Good luck!
Earlier quoted context omitted.
A lot of that group's publications are listed here, many involving optical flow: http://web.archive.org/web/20161014025823/http://gpu4vision.... Maybe this one from 2007: https://www-pequan.lip6.fr/~bereziat/cours/master/vision/pap...
Oh amazing find Icrs, thank you! :)
[0]: https://lmb.informatik.uni-freiburg.de/Publications/2010/Bro...
(Hierarchical) Optical flow is slow and prone to the need to manually adjust its constants for each scene (not really universal). Did you think about using 3D convolutions & attention with Deep Learning instead?
Back in 2008 there was an amazing demo of realtime dense optical flow on a GPU[0] but all the links are dead now. I've searched hard for another comparable implementation since then but had no luck. Does anyone have a hint on what technique they might have used? [0] https://www.youtube.com/watch?v=ssINeWRb58M
Back in 2008 there was an amazing demo of realtime dense optical flow on a GPU[0] but all the links are dead now. I've searched hard for another comparable implementation since then but had no luck. Does anyone have a hint on what technique they might have used? [0] https://www.youtube.com/watch?v=ssINeWRb58M
Really neat article. I have always wondered if these techniques are used at all in higher fps interpolations which always seem... off.