There is no need to learn SIMD. Just use proper vector_size attributes. Even without the fragile auto-vectorizer, the compiler uses SIMD then. Even with -O0
Everyone should know SIMD
251–260 of 263 posts
Re: Everyone should know SIMD
#252Earlier quoted context omitted.
A fourth alternative, in 64-bit systems, is to reserve a stupidly large chunk of memory up front with `mmap()` or equivalent (`malloc()` actually should work about as well). That way you guarantee that any extension will happen in place. There’s a limit to how much you can reserve, but since that limit is much higher than what you can actually use, you can make quite a few of those reservation before you run out of a…
In Rust at least, most things have a with_capacity( n ) constructor to ensure there's space for n elements (or n bytes, in the case of strings). I suppose there's no getting around the fact that if your collection has no known bounds, you'll have to do bounds checking + potential reallocation in the hot (push) path or risk having your program SIGSEGV. https://doc.rust-lang.org/std/?search=with_capacity
You can kinda do this in C++ too with `std::vector::reserve()`. The thing is if my collection has no known bounds, well, maybe this time it requires 64GB of space? Except RAM is at a premium and I only have 16GB right now, and no swap space to speak of. What am I gonna do?
The problem with overcommit, is that there is no reliable way to know in advance how much memory your program can actually use. That system with 16GB of RAM will allow you to "allocate" 128GB of memory. Just try it, call `malloc()` or the `with_capacity()` constructor, it will return a valid pointer or container. Try to write in that giant "buffer" at the beginning, at the end, somewhere in the middle at random… it will still work. Everything works exactly as if you really had a giant 128GB buffer, that you can write to and read back from…
…Until you hit somewhere below 5M different pages, where instead of getting a new page after the page fault, you get killed by the out of memory killer. With SIGTERM if you have the relevant privilege, so you have at least a chance of exiting cleanly, but user programs just get SIGKILL. No appeal, no way to check.
Well there is a way to check, kinda: allocate a fixed amount of memory up front, hit all the pages, and if your program didn't crash, it really has the amount of memory you just gave it. Then perform all your allocations within this real buffer you really have (this means a custom allocator). Any allocation success will be real, and bounds checks will actually work. It just doesn't play well with programs whose actual memory needs are highly unpredictable: most of the time you'll use much more memory than you need, and sometimes you won't have enough, forcing you to raise the threshold.
Re: Everyone should know SIMD
#253Earlier quoted context omitted.
Compilers are really good but really good is not actually that useful in cases where you need SIMD
I mean, utter bullshit. If you "need SIMD" you know exactly the programming pattern to guarantee SIMD from the compiler. And the single and only people who "need SIMD" know these rules. It is only the hobbyist "SIMD is neat" community that upvotes these ridiculous articles.
When I need SIMD I rarely use the compiler to help
Re: Everyone should know SIMD
#254Re: Everyone should know SIMD
#255Earlier quoted context omitted.
Eggs 11 years ago cost $0.89 (hell, TWO years ago!). Today they cost $5+.
Food inflation has been moderately high but egg prices were a mix of weird short term events and umm price fixing (look up the recent case). They are back to pre 2022 levels now! https://fred.stlouisfed.org/series/APU0000708111
I know that it was due to price fixing, but the cartelisation of many industries and lack of any serious anti-trust enforcement (or any other kind of white-collar enforcement) are major factors impacting the economy. A 3 million dollar fine is nothing but a slap on the wrist. Even the steeper fines on dairy producers haven't stopped them from consistently raising prices at the slightest provocation.
Re: Everyone should know SIMD
#256Good article! I just wouldn't start off with bold sentences as > SIMD can be simple to understand and > writing SIMD is just about as easy as a for loop and then the first example requires 12 lines to replace one line of scalar code. Be honest and say SIMD is hard but the results are worth it! (Another nitpick: if this article is for newbies, don't use SIMD-only words and concpts before explaining them. Step 5 is goo…
This is probably one of the biggest sins in technological teaching. Sure it is crucially important to take away the fear of a topic. But you don't do so by saying it is simple, you do so by showing it is simple. And it turns out sometimes you cannot show it is simple, because it is in fact very complex. But every complex topic is made up of smaller, simpler ones. Good teachers then manage to find a good order of thos…
Simple != easy
In fact getting things to be simple can often be very hard and cumbersome.
Re: Everyone should know SIMD
#257Earlier quoted context omitted.
Yep, I avoid saying the word simple almost entirely - its straight forward to get to the top of a mountain, it might still be incredibly arduous.
Simple Network Management Protocol Simple Mail Transfer Protocol Lightweight Directory Access Protocol Sometimes I think the RFC editors are trolling us.
SMTP is very simple, though.
Re: Everyone should know SIMD
#258Earlier quoted context omitted.
auto-vectorization is not nearly as good as you would hope it to be. The best SIMD optimizations likely require changing your data format from AoS to SoA.
While C++ may be reaching levels of Algol 68, PL/I complexity, with C++26 reflection you can do automatically. See https://github.com/cern-nextgen/reflmempp
That’s a research project. So no.
Re: Everyone should know SIMD
#259Earlier quoted context omitted.
Is there some way to write unit tests for cases where you know vectorisation should have been applied? I guess micro benchmarks should cover the performance part. We have ArchUnit to cover code structures, it would be nice if something similar exists for generated assembly.
I've been begging for years for a a [[must_vectorize]] annotation that I can place before a loop I care about, and turn it into a compile error if the compiler can't figure it out.
Just don't do it like Rust inlining annotations:
– Gently suggests inlining
#[inline] - Really suggests inlining
#[inline(always)] – Really Really suggests inlining (still not guaranteed!)