Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Falsehoods programmers believe about video

haasn.xyz

Post reply on HN
Print view View original

41–50 of 139 posts

Re: Falsehoods programmers believe about video

#41

Post by frozenport »

wstrange wrote: post #9
justinlaster wrote: post #6

> a H.264 hardware decoder can decode all H.264 files and > video decoding is easily parallelizable At a previous job, I don't know if it was just the field I was in or just bad luck, but having to explain this over and over again was kind of a personal nightmare. That being said, this is an excellent list!

Curious - Why is this? Does this assume streaming video, and you can't look ahead in the stream? If you can jump ahead, it would seem to be easy to have multiple threads, starting at key frames to decode the content. You'd have to splice them together, but this seems possible.

Decoding frames ahead of time gives no benifit to a user watching the video. The problem is how to decode a single frame in parallel. Contrary to the video expressed elsewhere, hardware decoders run a lot in parallel. As MultiCoreWare pointed out, one of the biggest challenges is latency.

Re: Falsehoods programmers believe about video

#42

Post by pedrocr »

the8472 wrote: post #38
pdkl95 wrote: post #29

Earlier quoted context omitted.

I believe[1] this isn't necessarily about broken files. There is a lot of variation allowed by the spec. One example that I've seen in the wild is extra-long (> 60 seconds) periods between I-frames. Seeking to an arbitrary point either requires searching backwards from the seek-point for an I-frame and storing a massive amount of RAM. As this usually isn't possible and would require decoding hundreds of frames, decod…

a 1-minute span for I-frames would not be prohibitive for parallel processing that the quotes part was referring to, with a 60-minute video it would still give you 60 segments to process in parallel.

A single uncompressed frame of 1080p video occupies 28MB in RAM, so 1 minute of 24fps video will take up 40GB. If you want to be able to run 4 cores at once it's 3 times that. You won't be doing that any time soon on your laptop or smartphone.

Re: Falsehoods programmers believe about video

#43

Post by jheriko »

derefr wrote: post #26

> rendering subtitles at the output resolution is better than rendering them at the video resolution I would like to know what's wrong with this approach. I watch a lot of commentated speed-run videos: that's often something like ~244p video, plus soft subtitles. The subtitles get rendered at the source resolution (presumably, into the video framebuffer) and then upscaled along with the image, forcing them to be a ti…

the point is precisely that it is more complicated than this obvious interpretation.

whilst i don't necessarily agree... i do agree that if you want to conform to specs then you can't go thinking this way.

Re: Falsehoods programmers believe about video

#44

Post by CoolGuySteve »

derefr wrote: post #26

> rendering subtitles at the output resolution is better than rendering them at the video resolution I would like to know what's wrong with this approach. I watch a lot of commentated speed-run videos: that's often something like ~244p video, plus soft subtitles. The subtitles get rendered at the source resolution (presumably, into the video framebuffer) and then upscaled along with the image, forcing them to be a ti…

It's also missing the most common error I see: conflating subtitles with closed captions.

Closed captions are positioned on the screen to indicate who's talking, have descriptive audio for sound effects, and should be in a high contrast easy to read font (most people with hearing deficiencies also have problems seeing, ie: out of date prescriptions for both hearing aids and eye glasses).

As far as I know, QuickTime does it right but the Apple TV, Netflix, and YouTube fuck it up, but that's because I helped write the QuickTime one way back.

Re: Falsehoods programmers believe about video

#45

Post by jheriko »

it is true, video is a nightmare mess littered with weird functionality nobody needs. (limited range only just disappeared in rec 2100, optionally??? really??? i'm not worried about my electron gun in my CRT from 1975 these days...nor do i want to know what a Y or a Cb or a Cr means because everything is RGB and B&W TV is long dead... and 4:2:2 is not exactly compression so much as computational overhead etc.. etc.)

its a nightmare, but the reason for these observations is precisely that it shouldn't be a nightmare. this area of programming is a wasteland ... nobody that good wants to solve these trivial problems :/

Re: Falsehoods programmers believe about video

#46

Post by jheriko »

wstrange wrote: post #9
justinlaster wrote: post #6

> a H.264 hardware decoder can decode all H.264 files and > video decoding is easily parallelizable At a previous job, I don't know if it was just the field I was in or just bad luck, but having to explain this over and over again was kind of a personal nightmare. That being said, this is an excellent list!

Curious - Why is this? Does this assume streaming video, and you can't look ahead in the stream? If you can jump ahead, it would seem to be easy to have multiple threads, starting at key frames to decode the content. You'd have to splice them together, but this seems possible.

seems to be easy, but each frame depends on previous frames... so now you need to share lots of data between threads. its not as embarrassingly parallel as it looks from a naive perspective.

although i contend that most decoders are very threadable - just that the people trying to do it usually lack the time or the skill, more usually the former.

the state of video in programming is a total mess from my experiences.

Re: Falsehoods programmers believe about video

#47

Post by unscaled »

mojuba wrote: post #20
inopinatus wrote: post #8

Earlier quoted context omitted.

Perhaps there is scope for a list of Falsehoods Programmers Believe About Falsehoods Programmers Believe.

Let's start then: 1. Everything said in every "Falsehoods Programmers Believe..." list is true. The Falsehoods sound like ultimate truths only because of the literary genre. They sound like they were written by an expert who not only knows what's true, but also knows what we think we know, which kind of automatically takes him/her to the next level of expertise.

3. Every falsehood that is true should be accounted for.

4. Every falsehood that is true CAN be accounted for.

5. Making your code compatible with a falsehood doesn't come with a price.

6. There are no falsehoods which are mutually exclusive.

Re: Falsehoods programmers believe about video

#48

Post by brigade »

the8472 wrote: post #13
saurik wrote: post #11

Earlier quoted context omitted.

1) You are now assuming that "seeking to a position will produce the same output as decoding to a position"; even if the video is well-formed (and you don't end up with massive issues where the key frames just don't work correctly) you are likely going to end up with subtle discontinuities between every segment. 2) You are now going to have to be buffering a couple seconds worth of uncompressed video somewhere, proba…

> 1) You are now assuming that "seeking to a position will produce the same output as decoding to a position"; even if the video is well-formed (and you don't end up with massive issues where the key frames just don't work correctly) you are likely going to end up with subtle discontinuities between every segment. Wouldn't "the keyframes just don't work correctly" result in corrupted output anyway? If we're worrying…

As an example, there exist bitstreams where there aren't actually any keyframes, but instead the encoder guarantees that the decoder output converges to correct after decoding some number of frames. It's actually kinda how MDCT audio codecs work; it's just very rare in video.

Re: Falsehoods programmers believe about video

#49

Post by haasn »

akiselev wrote: post #40
derefr wrote: post #26

> rendering subtitles at the output resolution is better than rendering them at the video resolution I would like to know what's wrong with this approach. I watch a lot of commentated speed-run videos: that's often something like ~244p video, plus soft subtitles. The subtitles get rendered at the source resolution (presumably, into the video framebuffer) and then upscaled along with the image, forcing them to be a ti…

I think that point should be amended to say "rendering subtitles at the output resolution is always better than rendering them at the video resolution." You don't want to upscale 244p soft subtitles to 1080p but you do want to default to giving video authors creative control over how the subtitles are displayed. The ASS subtitle format allows for some very complex styling that can be used as an artistic element in vi…

This comment is pretty much what I was going for. I've reworded it to make it clearer.

The issue you can run into in practice is stuff like softsubbed signs, which can clash and look out of place with the native video if you render them at full res. There's also a related issue, which is that if you're using something like motion interpolation (e.g. “smoothmotion”, “fluidmotion” etc. or even stuff like MVTools/SVP), softsubbed signs will not match the video during pans etc., making them stutter and look very out-of-place - the only way to fix that is to render them on top of the video before applying the relevant motion interpolation algorithms.

Personally I've always wished for a world in which subtitles are split into two files, one for dialogue and for signs, with an ability to distinguish between the two. (Heck, I think softsubbed signs should just be separate transparent video streams that are overlayed on top of the native picture, allowing you to essentially hardsub signs while still being capable of disabling them)

Also, sometimes, rendering at full resolution is prohibitively expensive, e.g. watching heavily softsubbed 720p content on a 4K screen.

Re: Falsehoods programmers believe about video

#50

Post by [deleted] »

derefr wrote: post #26

> rendering subtitles at the output resolution is better than rendering them at the video resolution I would like to know what's wrong with this approach. I watch a lot of commentated speed-run videos: that's often something like ~244p video, plus soft subtitles. The subtitles get rendered at the source resolution (presumably, into the video framebuffer) and then upscaled along with the image, forcing them to be a ti…

[deleted]
Post reply on HN