Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Guide to Adopting AV1 Encoding

bitmovin.com

Post reply on HN
Print view View original

41–50 of 95 posts

Re: Guide to Adopting AV1 Encoding

#41

Post by matsemann »

bick_nyers wrote: post #32
dylan604 wrote: post #6

Earlier quoted context omitted.

Tricks of the trade: why have one computer compress the file when you can split it up into logical segments and have each segment sent to its own encoder?

This is done particularly when you are implementing adaptive bitrates (the thing that Netflix uses where it automatically sends you a higher or lower quality picture depending on your Internet connection). In adaptive bitrate world, you split a video up into fragments, say 2-10 seconds large, and encode each segment in multiple bitrates, so that every say 5 seconds the video player can make a decision to download a d…

For certain containers / codecs you can concatenate files without re-encoding. Do it quite often with ffmpef using -c:copy and it's basically at the speed of the disk.

Re: Guide to Adopting AV1 Encoding

#42

Post by RealStickman_ »

sergiotapia wrote: post #11
jampekka wrote: post #2

Maybe this is not such a concern to audience of this article, but at least for me I'm stuck with h264 because VP9/AV1 encoding is really really slow. I'd love to use the codecs that are open and technically better, but when the video encodes at 1fps, it's too just too convenient to use the magnitudes faster h264. I'm probably not up to date with newer/hardware encoders and please let me know if my view is outdated.

How are you getting 1fps encoding? I see 332fps on my 12900K with preset 10. -vf scale=1280:720 -c:v libsvtav1 -crf 30 -preset 7 -c:a libopus -b:a 96k -ac 2

Preset 10 is pretty fast, but you're disabling a lot of the features that allow better quality and compression ratios (see readme on the svt-av1 repo). For archiving 3 or 4 is probably good.

Re: Guide to Adopting AV1 Encoding

#43

Post by scottlamb »

bick_nyers wrote: post #32
dylan604 wrote: post #6

Earlier quoted context omitted.

Tricks of the trade: why have one computer compress the file when you can split it up into logical segments and have each segment sent to its own encoder?

This is done particularly when you are implementing adaptive bitrates (the thing that Netflix uses where it automatically sends you a higher or lower quality picture depending on your Internet connection). In adaptive bitrate world, you split a video up into fragments, say 2-10 seconds large, and encode each segment in multiple bitrates, so that every say 5 seconds the video player can make a decision to download a d…

> Ok, but why not split the file up for standard encoding? Well, you can't just concatenate two .mp4 together without re-encoding and have it make sense to most media players (as far as I am aware)

You can't just literally `cat foo-[123].mp4 > foo.mp4` with old-school non-fragmented .mp4 files, but you just have to shuffle the container stuff around a bit. You don't need to re-encode.

One downside is if you decide ahead of time that you're going to divide the video into fixed 5-second fragments/segments/chunks to encode independently, you're going to end up with that-length closed GOPs that don't match scene transitions or the like. IDR frame every 5 seconds. So no B/P frames that reference stuff 10 seconds ago, no periodic incremental refresh, nothing fancy.

Re: Guide to Adopting AV1 Encoding

#44

Post by bick_nyers »

dylan604 wrote: post #37
bick_nyers wrote: post #32

Earlier quoted context omitted.

This is done particularly when you are implementing adaptive bitrates (the thing that Netflix uses where it automatically sends you a higher or lower quality picture depending on your Internet connection). In adaptive bitrate world, you split a video up into fragments, say 2-10 seconds large, and encode each segment in multiple bitrates, so that every say 5 seconds the video player can make a decision to download a d…

okay, i don't agree with anything in your reply. segmenting a video file for HLS/DASH delivery is not at all the same thing I'm suggesting. Just for the sake of round numbers, i'm saying to take a 90 minute feature into nine 10-minute segments. fire up 10 separate instances to encode each 10-minute segment. you've just increased the encode time 10x. also, DASH/HLS does not require segmented files. you can have a sing…

Why did the encode time increase by 10x in that instance? Can't you just seek in the video to the I-frame before the cut point and start your encode there?

I've never tried merging streams across computers so was naively just thinking that your output from each computer would be an MP4 but that makes sense.

I pulled that info. from a Netflix talk, perhaps video cameras back from when that talk occured didn't compress the video for you? Besides, isn't IMAX all intra-encoded? It was my understanding that IMAX films are actually just a series of J2K images, so I would imagine that the video cameras used there would also be intra-encoded.

Re: Guide to Adopting AV1 Encoding

#45

Post by bick_nyers »

matsemann wrote: post #41
bick_nyers wrote: post #32

Earlier quoted context omitted.

This is done particularly when you are implementing adaptive bitrates (the thing that Netflix uses where it automatically sends you a higher or lower quality picture depending on your Internet connection). In adaptive bitrate world, you split a video up into fragments, say 2-10 seconds large, and encode each segment in multiple bitrates, so that every say 5 seconds the video player can make a decision to download a d…

For certain containers / codecs you can concatenate files without re-encoding. Do it quite often with ffmpef using -c:copy and it's basically at the speed of the disk.

The parent comment I was responding to was discussing splitting encodes across multiple computers then re-combining which is what I was referring to. Still sounds like it is possible and I was incorrect.

Re: Guide to Adopting AV1 Encoding

#46

Post by Jap2-0 »

shmerl wrote: post #7

> YouTube for example, encodes content in H.264/AVC, VP9 and AV1 Why don't I see AV1 in many Youtube videos though? Checking with yt-dlp. It looks like they were planning to use it, but didn't really roll it out.

My (not super informed) understanding is that it may also depend on how popular the video is - the increase in encoding time is less worth it if there are few people seeing quality or bandwidth benefits (depending on how they tune it).

Re: Guide to Adopting AV1 Encoding

#47

Post by turtlebits »

jampekka wrote: post #2

Maybe this is not such a concern to audience of this article, but at least for me I'm stuck with h264 because VP9/AV1 encoding is really really slow. I'd love to use the codecs that are open and technically better, but when the video encodes at 1fps, it's too just too convenient to use the magnitudes faster h264. I'm probably not up to date with newer/hardware encoders and please let me know if my view is outdated.

Same here. I'm not willing to pony up money for hardware/compute to encode AV1.

I encode x265 on a $2.5/month VPS w/ 4gb ram during the off hours. AV1 is almost an order of magnitude slower.

Re: Guide to Adopting AV1 Encoding

#48

Post by padenot »

tombert wrote: post #25
dylan604 wrote: post #20

Earlier quoted context omitted.

h.264 is from the 90s, so of course it's fast after ~30 years of use. hell, when I first got into encoding, we had dedicated expansion cards to do MPEG-1/MPEG-2 encoding because it was so difficult at the time. New codecs always take time in the beginning while the encoding software is tweaked/optimized. Eventually, it becomes part of the CPU hardware and then we all make comments like "remember when ____ was so slow…

Oh definitely, no argument here, I'm 100% ok with AV1 becoming the standard "video codec to rule them all", but I'm saying that in the short term, it's difficult to recommend AV1 or VP9 over h264 (at least for personal use). H264 encodes 10x faster, still gives reasonably decent compression, is supported by basically every consumer device [1] and browser out of the box, and very soon will have all the patents for it…

Raspberry Pi have had hardware decoder for h264 for as long as they've existed (I think?), but dropped in the most recent version. I don't understand why.

They've recently contributed non-trivial patches to Firefox to use the embedded Linux API for video hardware acceleration (V4L2, vs. VAAPI on desktop that we also support), and are shipping the h264ify extension with their Firefox build to get that codec often for their users so that the experience is good on older devices.

Maybe the 5 is that much faster than it's not needed as much, but h264 represent so much content that it feels a bit surprising anyways.

But I'm just a software person, hardware is complicated differently.

Re: Guide to Adopting AV1 Encoding

#49

Post by querez »

brucethemoose2 wrote: post #4
jampekka wrote: post #2

Maybe this is not such a concern to audience of this article, but at least for me I'm stuck with h264 because VP9/AV1 encoding is really really slow. I'd love to use the codecs that are open and technically better, but when the video encodes at 1fps, it's too just too convenient to use the magnitudes faster h264. I'm probably not up to date with newer/hardware encoders and please let me know if my view is outdated.

The hardware encoders are very fast and generally better than x264 (but not by as much as you'd think with the x264 slow preset). In addition, there are fast threaded AV1 encoders you may be overlooking, like SVT-AV1. For non-realtime, my favorite is av1an, which also yields better quality than is possible from aomenc and works with pretty much any encoder/codec: https://github.com/master-of-zen/Av1an

av1an is essentially a "wrapper" around other encoders. I've played around with it lot in the past, but have never seen any real quality gains from using it (as measured by VMAF) instead of just using the encoder directly. Am I doing something wrong? What exactly does av1an buy me (except maybe for better independent threads?)

Re: Guide to Adopting AV1 Encoding

#50

Post by adgjlsfhk1 »

rb2k_ wrote: post #24
blihp wrote: post #18

Earlier quoted context omitted.

AV1 hardware encode just started rolling out with the most recent GPU architectures so the majority of hardware still has to do software encoding. I'd also guess that a very large fraction of the hardware out there doesn't even have hardware decode for AV1 since it was only the last generation of GPUs that got that. AV1 is mainly solving problems for the platform owners (google/netflix/facebook etc) and h264 will pro…

In the past (with h265 / h264 at least), hardware encoding always ended up with visibly worse quality (and often even bigger file sizes) compared to a software encoder like x264/x265. Do you happen to know if that's still the case? (I guess for use-cases such as live streaming it doesn't matter that much, but for video that ends up in some archive, it's probably less acceptable)

hardware encoding gives up a little quality and filesize, but hardware encoding of AV1 will generally beat software encoding of X264 on all axes.
Post reply on HN