FFmpeg devs boast of another 100x leap thanks to handwritten assembly code • Hacker News Board

Re: FFmpeg devs boast of another 100x leap thanks to handwritten assembly code

#131

Post by rerdavies » Fri, Jul 25, 2025, 5:41 AM UTC

the8472 wrote: post #98

rerdavies wrote: post #76

Earlier quoted context omitted.

Regular compilers are actually extraordinarily good at auto-vectorizing. There are a few oddities that one has to be aware of; but in my experience, if you offer a GCC or Clang compiler an opportunity to auto-vectorize, it will leap on it pretty ruthlessly. I have done a lot of work recently with auto-vectorizing code for high-performance realtime audio processing. And it's pretty extraordinary how good GCC and Clang…

Good at autovectorizing? Eh, if you massage your code in godbolt until it kicks in. But just naively writing code often falls off the happy path. And even when it autovectorizes it can sometimes produce such gems as vmovdqu xmm1, xmmword ptr [rdx] vmovdqu xmm0, xmmword ptr [rdx + 16] mov rax, rdi vmovd ecx, xmm1 or byte ptr [rsi], cl vpextrb ecx, xmm1, 1 or byte ptr [rsi + 1], cl vpextrb ecx, xmm1, 2 or byte ptr [rsi…

Right. That is a given. you massage your code in godbolt (or equivalent) until it kicks in. Not denying that. Compile but verify.

Re: FFmpeg devs boast of another 100x leap thanks to handwritten assembly code

#132

Post by rerdavies » Fri, Jul 25, 2025, 6:42 AM UTC

the8472 wrote: post #98

rerdavies wrote: post #76

Earlier quoted context omitted.

Regular compilers are actually extraordinarily good at auto-vectorizing. There are a few oddities that one has to be aware of; but in my experience, if you offer a GCC or Clang compiler an opportunity to auto-vectorize, it will leap on it pretty ruthlessly. I have done a lot of work recently with auto-vectorizing code for high-performance realtime audio processing. And it's pretty extraordinary how good GCC and Clang…

Good at autovectorizing? Eh, if you massage your code in godbolt until it kicks in. But just naively writing code often falls off the happy path. And even when it autovectorizes it can sometimes produce such gems as vmovdqu xmm1, xmmword ptr [rdx] vmovdqu xmm0, xmmword ptr [rdx + 16] mov rax, rdi vmovd ecx, xmm1 or byte ptr [rsi], cl vpextrb ecx, xmm1, 1 or byte ptr [rsi + 1], cl vpextrb ecx, xmm1, 2 or byte ptr [rsi…

Didn't require that much coaxing. -O3 --fast-math, i7-avx architecture. What did you do to break it?!

        xor     eax, eax
    .L2:
        vmovdqu xmm1, XMMWORD PTR [rdx+rax]
        vinsertf128     ymm1, ymm1, XMMWORD PTR [rdx+16+rax],   0x1
        vmovdqu xmm0, XMMWORD PTR [rsi+rax]
        vinsertf128     ymm0, ymm0, XMMWORD PTR [rsi+16+rax], 0x1
        vorps   ymm0, ymm0, ymm1
        vmovdqu XMMWORD PTR [rdi+rax], xmm0
        vextractf128    XMMWORD PTR [rdi+16+rax], ymm0, 0x1
        add     rax, 32
        cmp     rax, 4096
        jne     .L2
        vzeroupper
        ret

Re: FFmpeg devs boast of another 100x leap thanks to handwritten assembly code

#133

Post by rerdavies » Fri, Jul 25, 2025, 7:49 AM UTC

Const-me wrote: post #114

aa-jv wrote: post #94

Earlier quoted context omitted.

A good example is a vector addition kernel, which is simple, embarrassingly parallel, and well-suited for SIMD: void vector_add(float *a, float *b, float *c, int n) { for (int i = 0; i

Indeed, automatic vectorizers do such simple things pretty reliably these days. However, if you build your software from kernels like that, you leave a lot of performance on the table. For example, each core of my Zen 4 CPU at base frequency can add FP32 numbers with AVX1 or AVX-512 at 268 GB/sec, which results in 806 GB/sec total bandwidth for your kernel with two inputs and 1 output. However, dual-channel DDR5 memo…

On the other hand, enregistered value access is free; and L1 cache is .. what.. 2 cycles?

But you're right. It's hard to come up with enough computing to interleave with the actual expensive part, which is accessing memory. Even L2 cache isn't really fast enough to not be a bottleneck for typical vectorized operations.

If you look at TPU architectures, the the general pattern is: fast local large L1-cache-grade memory, preferably multi-ported, or multi-banked. Plus compute (whatever). The important bit being the memory architecture, not the compute. Plus blistering fast communication between cores over which results get streamed at speeds that are still not really fast enough.

Interestingly, I sat in on architecture meetings three decades ago, where Intel architects were privately telling us: "compute doesn't matter any more; it's all about memory speed".

Re: FFmpeg devs boast of another 100x leap thanks to handwritten assembly code

#134

Post by rerdavies » Fri, Jul 25, 2025, 8:04 AM UTC

rerdavies wrote: post #129

aa-jv wrote: post #94

Earlier quoted context omitted.

A good example is a vector addition kernel, which is simple, embarrassingly parallel, and well-suited for SIMD: void vector_add(float *a, float *b, float *c, int n) { for (int i = 0; i

OK. And which kernel function does that? Serious question.

Ok. Miscommunication. Different kernels. I was actually asking "which Linux kernel functions are going to benefit from auto-vectorization. There are not a lot of Linux kernel functions that take arguments that are two arrays of floating point values. Lots of string stuff that typically doesn't vectorize well (except in freakish cases), perhaps. But very very few functions that take arrays of floats as arguments. Yes, graphics libraries if you want to count those as kernel functions; but those primarily concern themselves with passing arrays of floats to co-processors, to be vectorized by a GPU (different problem).

As an interesting point of reference, the last time I did Windows kernel development (which was admittedly not recently), code running in ring 0 was not allowed to access SIMD registers because they weren't saved and loaded during kernel-code context switches. Not sure if that's the case, but it probably is. Context switches are a LOT faster if you don't have to save and load SIMD registers.

Re: FFmpeg devs boast of another 100x leap thanks to handwritten assembly code

#135

Post by rerdavies » Fri, Jul 25, 2025, 8:15 AM UTC

vlovich123 wrote: post #77

izabera wrote: post #46

Earlier quoted context omitted.

ffmpeg is not too different from a microbenchmark, the whole program is basically just: while (read(buf)) write(transform(buf))

> However, the developers were soon to clarify that the 100x claim applies to just a single function, “not the whole of FFmpeg.” So OP is correct. The 100x speed up is according to some misleading micro benchmark. The reason is that that transform is a huge amount of code and as OP said this will blow out the code cache while the amount of data you’re processing results in a blowout of the data cache. Net overall imp…

And it's probable that the developer is comparing code compiled at -O0 (no optimization) against hand-coded assembler, like they did the last time they claimed a 90x speed up.

So just to to summary: either a 100x, or a 100% speedup (depending on which source)

- comparing hand-coded assembler vs. unoptimized C code.

- on a function that was poorly written in the first place.

- in code that's so rarely used that nobody could be bothered to fix it for decades.

- and even then, a tiny function whose overall CPU cost was about 2% of CPU cost to perform the obsolete task that nobody cared about enough to fix.

- so basically code that fails the profile before optimize rule, and should never have been optimized in the first place.

I think that covers it.

Re: FFmpeg devs boast of another 100x leap thanks to handwritten assembly code

#136

Post by rerdavies » Fri, Jul 25, 2025, 8:18 AM UTC

dylan604 wrote: post #72

yieldcrv wrote: post #50

Earlier quoted context omitted.

> what it really means is it was extremely inefficient to begin with I care more about the outcome than the underlying semantics, to me thats kind of a given

Any average user would be more concerned with the end results. But this is a forum of not average users and more the people specifically that get off on the underlying semantics. Just look at how many people here are so infatuated with AI/LLMs and are so concerned about training data, models, number of tokens, yet completely gloss over the fact that every single one of these products will just make shit up. Those con…

Which seems an extremely odd point to make in a thread about a story that concerns itself with humans just making shit up.

Re: FFmpeg devs boast of another 100x leap thanks to handwritten assembly code

#137

Post by the8472 » Fri, Jul 25, 2025, 5:25 PM UTC

rerdavies wrote: post #132

the8472 wrote: post #98

Earlier quoted context omitted.

Good at autovectorizing? Eh, if you massage your code in godbolt until it kicks in. But just naively writing code often falls off the happy path. And even when it autovectorizes it can sometimes produce such gems as vmovdqu xmm1, xmmword ptr [rdx] vmovdqu xmm0, xmmword ptr [rdx + 16] mov rax, rdi vmovd ecx, xmm1 or byte ptr [rsi], cl vpextrb ecx, xmm1, 1 or byte ptr [rsi + 1], cl vpextrb ecx, xmm1, 2 or byte ptr [rsi…

Didn't require that much coaxing. -O3 --fast-math, i7-avx architecture. What did you do to break it?! xor eax, eax .L2: vmovdqu xmm1, XMMWORD PTR [rdx+rax] vinsertf128 ymm1, ymm1, XMMWORD PTR [rdx+16+rax], 0x1 vmovdqu xmm0, XMMWORD PTR [rsi+rax] vinsertf128 ymm0, ymm0, XMMWORD PTR [rsi+16+rax], 0x1 vorps ymm0, ymm0, ymm1 vmovdqu XMMWORD PTR [rdi+rax], xmm0 vextractf128 XMMWORD PTR [rdi+16+rax], ymm0, 0x1 add rax, 32…

It was an optimization bug in llvm. https://godbolt.org/z/GqdPGarsn We filed an issue, this one is fixed now. https://github.com/llvm/llvm-project/issues/65763

Re: FFmpeg devs boast of another 100x leap thanks to handwritten assembly code

#138

Post by Const-me » Fri, Jul 25, 2025, 5:30 PM UTC

rerdavies wrote: post #133

Const-me wrote: post #114

Earlier quoted context omitted.

Indeed, automatic vectorizers do such simple things pretty reliably these days. However, if you build your software from kernels like that, you leave a lot of performance on the table. For example, each core of my Zen 4 CPU at base frequency can add FP32 numbers with AVX1 or AVX-512 at 268 GB/sec, which results in 806 GB/sec total bandwidth for your kernel with two inputs and 1 output. However, dual-channel DDR5 memo…

On the other hand, enregistered value access is free; and L1 cache is .. what.. 2 cycles? But you're right. It's hard to come up with enough computing to interleave with the actual expensive part, which is accessing memory. Even L2 cache isn't really fast enough to not be a bottleneck for typical vectorized operations. If you look at TPU architectures, the the general pattern is: fast local large L1-cache-grade memor…

> L1 cache is .. what.. 2 cycles?

On Zen 4 CPU, I believe the typical latency of L1D is 4 cycles. However, if you (or your compiler) write AVX code which adds these floats, will still bottleneck on memory even if both inputs are in L1D cache. Each Zen 4 core can sustain two vaddps instructions per cycle, two loads per cycle, and one store per cycle. Due to the load and store bottlenecks, that kernel will only do one vaddps per cycle i.e. will waste 50% of theoretically available compute power.

> Intel architects were privately telling us: "compute doesn't matter any more; it's all about memory speed"

To be fair, that’s only true for automatically vectorized code, kernels like in the GP’s example. With sufficient efforts spent on software development, for some practical problems it’s possible to write codes which do saturate compute.

An example of such problem is multiplication of large matrices. A carefully written manually vectorized implementation should bottleneck on compute not memory, because theoretically required memory bandwidth scales as N^2, while theoretically required FLOPs scale as N^3 where N is size of the matrix.

That’s precisely what many BLAS libraries are doing under the hood. For the same reason GPU vendors report ridiculously high numbers of theoretical TFlops when multiplying low precision matrices with these special AI blocks, wmma/mfma instructions on AMD, tensor cores on nVidia.