How does a video codec work?
github.com
How does a video codec work?
1–10 of 53 posts
Re: How does a video codec work?
#2Playing videos is so complicated.
Re: How does a video codec work?
#3Good. Want more people to understand how compression works.
Things like motion estimation, encoding the differences in frames, all that stuff is magic to the uninitiated. :) But this stuff is absolutely worth learning and every engineer should show interest :)
Re: How does a video codec work?
#4One thing that I wish more folks did with DSP/media tech was to start with concepts instead of diving into details first.
Like, "how does a video codec work?" should start with: what is the problem? (reducing the bits per second required for a video stream, since it's big) and how? (don't send superfluous detail, since it's either redundant or imperceptible).
Than dive into the details of how color is represented, how images are structured, how 2d signal transforms work, the principle behind the DCT as a method of representing the same data with better energy compaction by decorrelating different components, and how that can be used advantageously to reduce the number of bits for a still image, then talk about shared data between images, etc etc.
I've noticed as a DSP guy that when I talk about things without concepts first that everyone's eyes glaze over. Although it is nice that everyone thinks it's black magic, good job security.
Re: How does a video codec work?
#5One thing that I wish more folks did with DSP/media tech was to start with concepts instead of diving into details first. Like, "how does a video codec work?" should start with: what is the problem? (reducing the bits per second required for a video stream, since it's big) and how? (don't send superfluous detail, since it's either redundant or imperceptible). Than dive into the details of how color is represented, ho…
That was great. Do you have a blog series sonewhere?
Re: How does a video codec work?
#6magic. got it.
Re: How does a video codec work?
#7If you're interested in more "hands on" or practical video codec learning, I recommend writing a H.261 decoder. Only two supported frame sizes, no B-frames, and no intra prediction (effectively a subset of JPEG) make for a simple yet possibly quite rewarding weekend project that can be completed in around 700 lines of C (my attempt). Unfortunately not much existing media is available in 261, but I think watching a video being decoded entirely by code you wrote is a pretty fun experience, including all the weird and amusing distortions you can see when debugging; and from there you can move on to MPEG-1 with variable frame sizes and B frames (another weekend, assuming you reuse much of the 261 exercise --- I ended up with 1k lines total to decode MPEG-1), and that has somewhat more existing media you'll be able to watch.
Then you can try H.262/MPEG-2 and enjoy the intricacies of handling interlacing as well as being able to decode DVDs and a lot of existing content; and then there's H.263 which has intra prediction... I haven't gotten past the first two largely for reasons of time and other things to play with, but IMHO getting a basic implementation of a video decoder is not that hard especially when you're working from a standard.
Re: How does a video codec work?
#8One thing that I wish more folks did with DSP/media tech was to start with concepts instead of diving into details first. Like, "how does a video codec work?" should start with: what is the problem? (reducing the bits per second required for a video stream, since it's big) and how? (don't send superfluous detail, since it's either redundant or imperceptible). Than dive into the details of how color is represented, ho…
Yeah, this article had an odd progression. With the premise "How does a codec work?", it goes (Make a distinction between codec and container) -> (Discuss several codecs and their brief history and royalty scheme) -> (Use heavy industry jargon to discuss details that only someone who already gets compression could possibly understand).
Re: How does a video codec work?
#9One thing that I wish more folks did with DSP/media tech was to start with concepts instead of diving into details first. Like, "how does a video codec work?" should start with: what is the problem? (reducing the bits per second required for a video stream, since it's big) and how? (don't send superfluous detail, since it's either redundant or imperceptible). Than dive into the details of how color is represented, ho…
I wish this about so many things. Instead of telling me that I should use some technology, tell me what problem it solves. How does the technology solve that problem?
You should use React. Why? What problem does it solve? How does it solve that problem? What did people do before React? When is React not the appropriate solution, or an overkill solution?
Re: How does a video codec work?
#10One thing that I wish more folks did with DSP/media tech was to start with concepts instead of diving into details first. Like, "how does a video codec work?" should start with: what is the problem? (reducing the bits per second required for a video stream, since it's big) and how? (don't send superfluous detail, since it's either redundant or imperceptible). Than dive into the details of how color is represented, ho…
To be fair the article does that. The above URL just links to an anchor in the middle where it talks about codec implementation. For instance this quote from the article
"We learned that it's not feasible to use video without any compression; a single one hour video at 720p resolution with 30fps would require 278GB*. Since using solely lossless data compression algorithms like DEFLATE (used in PKZIP, Gzip, and PNG), won't decrease the required bandwidth sufficiently we need to find other ways to compress the video.
...
In order to do this, we can exploit how our vision works. We're better at distinguishing brightness than colors, the repetitions in time, a video contains a lot of images with few changes, and the repetitions within the image, each frame also contains many areas using the same or similar color."
https://github.com/leandromoreira/digital_video_introduction...