John Carmack
7 hrs ·
Adventures with ffmpeg and color ranges.
A video legacy issue that can cause a lot of problems is the issue of “limited” versus “full” component range. For arcane reasons related to TV broadcast limitations, many video formats restrict the YUV color components to be in the 16..235 or 16..240 range instead of the full 0..255 range. Losing 14% of the already-barely-enough 8 bit dynamic range is bad enough, but it also often results in the black range starting at a quite visible grey value because most players don’t rescale the range. This is usually visible as ugly banding or blocking in dark scenes.
For ffmpeg, the trick to avoid this is to use ‘-pix_fmt yuvj420p’, which says to use the j-for-jpeg full range in a 420 YUV subsampled p-for-planar format.
If you are starting with either RGB images, a 10/12 bit format, or a yuvj420p format video as input, then with the libx264 codec, you would get a full range output. Note that any video processing tools used along the way could also limit the range, and once it is gone, there is no getting it back, so you must be very careful and check your entire pipeline!
When you us this format, ffmpeg complains about ‘deprecated pixel format used, make sure you did set range correctly’, but you should ignore this warning.
Ffmpeg would like the world to move to specifying the range independently from the YUV channel subsampling and layout:
Setting ‘-color_range 0 -pix_fmt yuv420p’ makes the output format yuv420p
Setting ‘-color_range 1 -pix_fmt yuv420p’ makes the output format yuv420p(tv)
Setting ‘-color_range 2 -pix_fmt yuv420p’ makes the output format yuv420p(pc)
Unfortunately, this isn’t yet uniformly handled throughout all the internal format tests.
The libx265 integration in ffmpeg didn’t support the deprecated yuvj420p pixel format, only the basic yuv420p one, and no matter what I did, my test videos were always coming out limited range. It didn’t matter if you add a ’-color_range 2’, or ‘-x265-params range=full’. Those will change the settings in the VUI (Video Usability Information) section of the output, but the values are still compressed to the limited range.
Regardless of the input data, any 8 bit h265 video coming out of ffmpeg was limited range!
I walked through the libx265 code looking for range compression, but it turned out that all the damage was being done by ffmpeg before it got to x265. Ffmpeg will automatically convert formats when the output differs from the input, and since x265 only supported yuv420p, any full range input will be processed.
Adding ‘-v 48’ to ffmpeg will dump more information, including this auto_scaler invocation, which is what is killing the full color range:
[auto_scaler_0 @ 000001f55ecd1a40] w:2048 h:2048 fmt:bgr24 sar:0/1 -> w:2048 h:2048 fmt:yuv420p sar:0/1 flags:0x4
I was about to start hacking the code to at least do what I wanted for my use case, but I tried an appeal to Twitter:
https://twitter.com/ID_AA_Carmack/status/1131715388067274753
My suspicions were confirmed, but Derek Buitenhuis went ahead and submitted an official patch to get yuvj420p accepted by libx265, and windows builds are already available at https://ffmpeg.zeranoe.com/builds/.
I suspect there was probably some way of working around this involving explicit format conversion filters with -src_range and -dst_range overrides, but this is now working as you would expect it:
ffmpeg -i source.mp4 -c:v libx265 -pix_fmt yuvj420p dest.mp4
Bravo to ffmpeg!