Live data from Hacker News

How does a video codec work?

github.com

21–30 of 53 posts

Re: How does a video codec work?

#21
I wish there was an article on how hardware decoding works. Detailed but approachable.

I guess reading source code of drivers and popular open-source video players will give me an idea, but it is too much work. Does anyone have a technical article highlighting differences between hardware decoders and important details.

Re: How does a video codec work?

#23

As much as I like Apple it seems like they are in the back pocket of big media when it comes to this topic. AFAICT Apple only supports h.264 and h.265 meaning 1.3 billion iOS devices have browsers that can't access open standards for video. Yet one more reason why Apple should be required to allow alternate browser engines IMO. Some will claim it's a battery issue but Apple has the resources to add hardware support f…

Not sure what you mean, a stock iPhone plays a VP8 webrtc stream with no issues?

https://www.webmfiles.org/demo-files/

Play in Chrome (Mac/Win/Linux/Android) and Firefox (Mac/Win/Linux/Android) but not Safari (Mac/iOS).

Here's a few more

https://commons.wikimedia.org/w/index.php?search=webm

I would be nice if Apple would support open standards. It's unclear what their motive is for not supporting them.

Re: How does a video codec work?

#24

As much as I like Apple it seems like they are in the back pocket of big media when it comes to this topic. AFAICT Apple only supports h.264 and h.265 meaning 1.3 billion iOS devices have browsers that can't access open standards for video. Yet one more reason why Apple should be required to allow alternate browser engines IMO. Some will claim it's a battery issue but Apple has the resources to add hardware support f…

Apple has made a lot of practical pushes on this topic and was heavily engaged in the HTML5 standard codec debate. Wiki has a good writeup of it: https://en.wikipedia.org/wiki/HTML5_video#Supported_video_an....

If I remember, they were basically arguing against Google as it favored codecs that put costs on consumers, not the servers? Anyone remember this?

I think video patents have pretty much patented all the basics making new standards nearly impossible. So, organizations are stuck waiting to be sued if they adopt a codec. Apple seems to be waiting.

Re: How does a video codec work?

#25

As much as I like Apple it seems like they are in the back pocket of big media when it comes to this topic. AFAICT Apple only supports h.264 and h.265 meaning 1.3 billion iOS devices have browsers that can't access open standards for video. Yet one more reason why Apple should be required to allow alternate browser engines IMO. Some will claim it's a battery issue but Apple has the resources to add hardware support f…

Both H.264 and H.265 are Open Standards, even more so than Open Media Alliance. They are just not Royalty Free.

And AFAICT, All Smartphone shipped in the past 3 - 4 years has had Hardware H.264 encoder and decoder enabled. That is roughly 4.5 Billion of Smartphone and virtually all Smartphone users has H.264 decoder.

H.264 Video distribution over the Web is also Royalty Free in perpetuity.

H.265 is a bag of hurt, hopefully H.266, the MC-IF [1] will sort this out before release. The only major company missing from that list is Qualcomm.

I dont mind Royalty, there are huge amount and Research and Investment going each next gen codec. The problem is when they start being greedy. If they just collect $0.2 per hardware encoder & decoder shipped, that is a $400M annual revenue stream form the codec, or $4B to $8 Billion over its Codec lifetime split across different companies.

[1] https://www.mc-if.org

Re: How does a video codec work?

#27
post #11

One thing that I wish more folks did with DSP/media tech was to start with concepts instead of diving into details first. Like, "how does a video codec work?" should start with: what is the problem? (reducing the bits per second required for a video stream, since it's big) and how? (don't send superfluous detail, since it's either redundant or imperceptible). Than dive into the details of how color is represented, ho…

Well DSP theory can make many engineering students eye's glaze over. Video compression theory could be explained to a fairly bright middle schooler.

How so? I took 3-4 DSP classes in university and none of them seemed particularly earth shattering. I think statistics and mathematic notation are a lot harder for students to work with. That said, information theory, which I would not consider a DSP course, was the hardest class I took. It requires mathematician’s math. None of that easy engineering math.

Re: How does a video codec work?

#28

If you're interested in more "hands on" or practical video codec learning, I recommend writing a H.261 decoder. Only two supported frame sizes, no B-frames, and no intra prediction (effectively a subset of JPEG) make for a simple yet possibly quite rewarding weekend project that can be completed in around 700 lines of C (my attempt). Unfortunately not much existing media is available in 261, but I think watching a vi…

Very cool idea! Is your code for this online anywhere?

Re: How does a video codec work?

#29
post #3

Good. Want more people to understand how compression works. Things like motion estimation, encoding the differences in frames, all that stuff is magic to the uninitiated. :) But this stuff is absolutely worth learning and every engineer should show interest :)

Why is it worth learning? I have a long list of stuff that I want to learn but codecs don't seem useful enough to be included there.

Codecs teach you if the high fidelity of the data you store may not be as important as you'd think it is, and sometimes you'd be able to process all of that with a fraction of the compute power and cost involved.

Re: How does a video codec work?

#30

Earlier quoted context omitted.

Not sure what you mean, a stock iPhone plays a VP8 webrtc stream with no issues?

The question is if there is a hardware based decoder supporting this. With a software decoder it will play just fine but it will consume a lot of resources, eventually draining the battery.

At least Apple A10 onwards have an hardware VP8 decoder, so...
Post reply on HN