Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Optimizing your programs for Arm platforms

community.arm.com

Post reply on HN
Print view View original

11–20 of 30 posts

Re: Optimizing your programs for Arm platforms

#11

Post by kimixa »

dzaima wrote: post #10
astrange wrote: post #2

This isn't a good article. I would say that if you're trying to rely on `restrict` and autovectorization you're doomed and should write it yourself. Even if it works on one compiler version, it won't work on all of them. (It could possibly work in a language that isn't C and is designed for it; Fortran or shader programs are easier to autovectorize, and something like ISPC starts out "vectorized" and gets "autoscalar…

Any modern compiler that bothers should be able to autovectorize most practical vectorizable things without issue, even without restrict. Of course there'll be some small inefficiencies or failed autovectorization sometimes, but small and big missed optimizations are in no way at all a problem unique to vectorization, so it's a moot point here.

"restrict" is getting around language issues where it's easy to "accidently" block things like vectorization, or having to reload values multiple times just in case pointers aliased. There's no compiler in the world that can work around this without more guarantees on the expected behavior from the programmer, as it would be incorrect according to the spec.

I've seen "obvious" big wins missed without it.

Re: Optimizing your programs for Arm platforms

#12

Post by dzaima »

kimixa wrote: post #11
dzaima wrote: post #10

Earlier quoted context omitted.

Any modern compiler that bothers should be able to autovectorize most practical vectorizable things without issue, even without restrict. Of course there'll be some small inefficiencies or failed autovectorization sometimes, but small and big missed optimizations are in no way at all a problem unique to vectorization, so it's a moot point here.

"restrict" is getting around language issues where it's easy to "accidently" block things like vectorization, or having to reload values multiple times just in case pointers aliased. There's no compiler in the world that can work around this without more guarantees on the expected behavior from the programmer, as it would be incorrect according to the spec. I've seen "obvious" big wins missed without it.

Both gcc and clang check for aliasing at runtime if not provable statically for autovectorization (granted, that can fail if you have reverse/strided/gather addresses, but those are less common; and yeah it does lead to some constant overhead, though likely not significant often).

Of minor note is that you can add "#pragma clang loop vectorize(assume_safety)" or "#pragma GCC ivdep" on the respective compilers to a loop to allow them to vectorize anything, which in my experience is much more functional than restrict. And even the vast majority of benefit I've gotten from it was just removing the alias check overhead (though it did catch a case of a reversing loop failing to vectorize due to a 32-bit index variable or something)

Re: Optimizing your programs for Arm platforms

#13

Post by astrange »

dzaima wrote: post #12
kimixa wrote: post #11

Earlier quoted context omitted.

"restrict" is getting around language issues where it's easy to "accidently" block things like vectorization, or having to reload values multiple times just in case pointers aliased. There's no compiler in the world that can work around this without more guarantees on the expected behavior from the programmer, as it would be incorrect according to the spec. I've seen "obvious" big wins missed without it.

Both gcc and clang check for aliasing at runtime if not provable statically for autovectorization (granted, that can fail if you have reverse/strided/gather addresses, but those are less common; and yeah it does lead to some constant overhead, though likely not significant often). Of minor note is that you can add "#pragma clang loop vectorize(assume_safety)" or "#pragma GCC ivdep" on the respective compilers to a lo…

> Both gcc and clang check for aliasing at runtime if not provable statically for autovectorization

This is often enough to make it unworkable, because it means you're inserting checks into hot loops.

Also, if you partially vectorize something yourself you have to write similar setup code, which might involve scalar versions of the loop, but then autovectorization can come by and vectorize those, so now you have duplicate setup code making it worse than nothing.

Re: Optimizing your programs for Arm platforms

#14

Post by astrange »

dzaima wrote: post #10
astrange wrote: post #2

This isn't a good article. I would say that if you're trying to rely on `restrict` and autovectorization you're doomed and should write it yourself. Even if it works on one compiler version, it won't work on all of them. (It could possibly work in a language that isn't C and is designed for it; Fortran or shader programs are easier to autovectorize, and something like ISPC starts out "vectorized" and gets "autoscalar…

Any modern compiler that bothers should be able to autovectorize most practical vectorizable things without issue, even without restrict. Of course there'll be some small inefficiencies or failed autovectorization sometimes, but small and big missed optimizations are in no way at all a problem unique to vectorization, so it's a moot point here.

You need restrict for a loop involving char* or equivalent because it can alias any type in C. Without it, the compiler has to preserve memory accesses exactly, and then it's hardly able to do any optimizations. If you're writing all the memory accesses out perfectly optimally already, then you don't need the compiler for much.

Re: Optimizing your programs for Arm platforms

#15

Post by astrange »

SubjectToChange wrote: post #9
astrange wrote: post #8

Earlier quoted context omitted.

> Projects like ffmpeg are able to do it because they're pulling from a massive pool of contributors. It has the opposite problem; it's drawing from a small pool of skilled contributors, because not enough people have learned it, because so much other incorrect advice thinks it's fine to use autovectorization that doesn't work. > Honestly, who is saying that? The recent article here about ffmpeg's use of assembly exc…

It has the opposite problem; it's drawing from a small pool of skilled contributors,.. The project has 2000+ direct contributors and even more indirect contributors on its mailing lists. ...because so much other incorrect advice thinks it's fine to use autovectorization that doesn't work. There are few high performance programmers who genuinely believe that autovectorization can compete with hand written assembly. Th…

> The project has 2000+ direct contributors and even more indirect contributors on its mailing lists.

I'm one of them, so please just believe me instead of trying to correct me ;) It's an ongoing problem the project talks about that there aren't enough newcomers ready to write more SIMD code with good enough quality.

> Should they be showering ffmpeg et al. with praise or something?

The top reply is "just do this other thing that the article said was unworkable", so not doing that would be a start. Though, the article could've spent some more time explaining why intrinsics don't work well enough.

> but hardly anyone else can justify doing that

Other people mostly only target one CPU architecture as they're less important, but they also get paid and ffmpeg developers largely didn't. (These days more of them do, but those people are contributing security work more than performance work I think.)

It's similar to how x264 was better than every commercial competitor while working for free, simply because they took more time to think about what they were doing.

Re: Optimizing your programs for Arm platforms

#16

Post by neonsunset »

SubjectToChange wrote: post #9
astrange wrote: post #8

Earlier quoted context omitted.

> Projects like ffmpeg are able to do it because they're pulling from a massive pool of contributors. It has the opposite problem; it's drawing from a small pool of skilled contributors, because not enough people have learned it, because so much other incorrect advice thinks it's fine to use autovectorization that doesn't work. > Honestly, who is saying that? The recent article here about ffmpeg's use of assembly exc…

It has the opposite problem; it's drawing from a small pool of skilled contributors,.. The project has 2000+ direct contributors and even more indirect contributors on its mailing lists. ...because so much other incorrect advice thinks it's fine to use autovectorization that doesn't work. There are few high performance programmers who genuinely believe that autovectorization can compete with hand written assembly. Th…

Yes, I support your sentiment.

On the topic, in C# (*), it is currently strongly recommended to rewrite any code that used to rely on specific ISA and ISA extensions (SSE4.2, AVX, NEON) to cross-platform methods on `Vector128/256/512` and `Vector` themselves.

Also, .NET is getting support for SVE2 now on top of `Vector`, which is nice.

* Which has proper portable SIMD API, unlike Java panama vectors in their current shape (codegen and API limitations makes them currently an unsuitable counterpart)

Re: Optimizing your programs for Arm platforms

#17

Post by astrange »

neonsunset wrote: post #16
SubjectToChange wrote: post #9

Earlier quoted context omitted.

It has the opposite problem; it's drawing from a small pool of skilled contributors,.. The project has 2000+ direct contributors and even more indirect contributors on its mailing lists. ...because so much other incorrect advice thinks it's fine to use autovectorization that doesn't work. There are few high performance programmers who genuinely believe that autovectorization can compete with hand written assembly. Th…

Yes, I support your sentiment. On the topic, in C# (*), it is currently strongly recommended to rewrite any code that used to rely on specific ISA and ISA extensions (SSE4.2, AVX, NEON) to cross-platform methods on `Vector128/256/512 ` and `Vector ` themselves. Also, .NET is getting support for SVE2 now on top of `Vector `, which is nice. * Which has proper portable SIMD API, unlike Java panama vectors in their curre…

Is it strongly recommended by people who've successfully written performant software on multiple platforms (like ffmpeg), or by compiler engineers? Because this is the kind of thing compiler engineers would like to believe is true, but in practice isn't unless you have the power to make them fix it for you.

There are actual differences between ISAs here and if your case deoptimizes on one of them then there wasn't a point in using SIMD at all.

(Examples: whether it has a full permute, whether unaligned loads are fast or unusably slow, whether it supports half-floats.)

And of course nobody can do an abstraction for MMX, so they just pretend it doesn't exist.

Re: Optimizing your programs for Arm platforms

#18

Post by janwas »

astrange wrote: post #8
SubjectToChange wrote: post #7

Earlier quoted context omitted.

Writing good assembly is a niche skill, especially SIMD assembly. Projects like ffmpeg are able to do it because they're pulling from a massive pool of contributors. In general writing raw assembly should be avoided unless you're genuinely in a position of knowing better. ...people constantly replying "um actually you never need to write anything in assembly" to them. Honestly, who is saying that?

> Projects like ffmpeg are able to do it because they're pulling from a massive pool of contributors. It has the opposite problem; it's drawing from a small pool of skilled contributors, because not enough people have learned it, because so much other incorrect advice thinks it's fine to use autovectorization that doesn't work. > Honestly, who is saying that? The recent article here about ffmpeg's use of assembly exc…

> it doesn't use intrinsics because they aren't actually easier to work with; they are not faster, not more portable

Well golly, I'll just have to disagree based on >20 years of experience, including several in assembly. asm is only (maybe) faster for the code we manage to get written.

From where I sit, video codecs are a rare special case in that the format is standardized, changes only every few years, and has only a few but super-time-critical kernels. For many many other use cases, the situation looks different and productivity matters more. Would you rather get a 10x speedup on 40% of the cycles, or 8x on 80%?

BTW the "not more portable" comment is a strawman because intrinsics themselves indeed aren't portable, but a wrapper library on top (such as our Highway) is.

Re: Optimizing your programs for Arm platforms

#19

Post by BoingBoomTschak »

astrange wrote: post #2

This isn't a good article. I would say that if you're trying to rely on `restrict` and autovectorization you're doomed and should write it yourself. Even if it works on one compiler version, it won't work on all of them. (It could possibly work in a language that isn't C and is designed for it; Fortran or shader programs are easier to autovectorize, and something like ISPC starts out "vectorized" and gets "autoscalar…

The big problem is that gcc/clang don't seem to have a concept of optimization notices, like SBCL does.

Nobody is more appropriate than the compiler to warn you that it couldn't optimize something costly and why.

Re: Optimizing your programs for Arm platforms

#20

Post by neonsunset »

astrange wrote: post #17
neonsunset wrote: post #16

Earlier quoted context omitted.

Yes, I support your sentiment. On the topic, in C# (*), it is currently strongly recommended to rewrite any code that used to rely on specific ISA and ISA extensions (SSE4.2, AVX, NEON) to cross-platform methods on `Vector128/256/512 ` and `Vector ` themselves. Also, .NET is getting support for SVE2 now on top of `Vector `, which is nice. * Which has proper portable SIMD API, unlike Java panama vectors in their curre…

Is it strongly recommended by people who've successfully written performant software on multiple platforms (like ffmpeg), or by compiler engineers? Because this is the kind of thing compiler engineers would like to believe is true, but in practice isn't unless you have the power to make them fix it for you. There are actual differences between ISAs here and if your case deoptimizes on one of them then there wasn't a…

The quote reads as "In C#, it is recommended..."

Which means that, in the past, C# did not have cross-platform SIMD abstractions aside from a very limited set of arithmetic operations on Vector so that manually vectorized code had to rely on SIMD intrinsics introduced earlier (AVX2, AdvSimd, etc.).

However, .NET 7 introduced a set of common arithmetic, logical and bitwise operations on Vector128/256/512[0] (VectorXXX are common vector width types that used to be consumed by intrinsic APIs exclusively), and subsequently both 7 and 8 also included QoL improvements for more high-level Vector.

This change rendered the code that duplicated SIMD paths per-platform mostly obsolete save for certain operations like `Shuffle` (which then got addressed by introducing ShuffleUnsafe which is just a raw platform-specific shuffle with the expectation that the users will account for those manually, or the set of outputs they care about has sufficiently common behavior everywhere).

CoreLib itself relies on these APIs now and there is no reason to use platform-specific intrinsics in most situations over cross-platform API.

Now, one of the reasons .NET can do this is because it does not have to target such a wide range of platforms regular C code has to deal with: most code out there only ever cares about x86, x86_64 (multiple flavors due to SSE2/4, AVX/2 and AVX512), armv7, armv8a and now also wasm (with packed SIMD), and maybe riscv in the future. Out of those, x86_x64 and armv8a receive most attention and performance investment, which are sufficiently similar save for movemask workhorse emulation of which is suboptimal (community has learned vshrn[1] and other tricks for common operations since then to avoid the issue).

With that said, C++ has its own experimental cross-plat SIMD abstraction, and there are many high-quality frameworks that allow to abstract away writing SIMD code manually completely. There is also a Rust crate[2] that offers C#-style SIMD abstraction, so it's not exclusive to C# (or particularly difficult in systems programming languages) but C# is probably the one and only high-level language to offer it with an assurance to emit good codegen (unless you abuse it too hard).

[0] https://github.com/dotnet/runtime/blob/main/docs/coding-guid...

[1] https://github.com/U8String/U8String/blob/main/Sources/U8Str...

[2] https://github.com/Lokathor/wide

Post reply on HN