71–80 of 263 posts
99% of developers should just ignore SIMD. Most projects have a lot of low hanging fruit to increase performance, and still nobody finds the time to solve them.
That doesn't mean you should default to a slower implementation for new code. If you do, you're just creating more low-hanging fruit that nobody will find time to solve.
To bolster the argument, even if you do not plan to write the SIMD yourself or will "just get AI to do it", it is important to know what can be fast in SIMD (and on what hardware). That allows you to design your algorithms and structure your code so that the SIMD is possible. Internalizing things like how data dependencies matter, how expensive it is to increase the width of your vector elements (and how to avoid the…
> what can be fast I think this doesn't get talked about enough. If your input is a big run of data that is being checked/transformed in one shot, it works well. But, if you're likely to have to make a decision on several bytes of the input, SIMD will be the same or slower than the scalar method. It's not a magic "go fast" button.
99% of developers should just ignore SIMD. Most projects have a lot of low hanging fruit to increase performance, and still nobody finds the time to solve them.
That doesn't mean you should default to a slower implementation for new code. If you do, you're just creating more low-hanging fruit that nobody will find time to solve.
Earlier quoted context omitted.
Yeah, data layout/cache aware layouts are really key if you really want to unlock making something that ends up in a hot loop fast with SIMD. Also, avoiding allocations or vtable lookups or a lot of indirection in the part of the code that's actually "hot" is really important. Vectors (in C++) at least aren't necessarily the best fit either, if you end up doing anything that can call an allocation unexpectedly.
> Vectors (in C++) at least aren't necessarily the best fit either I'm not sure if you use a different allocation strategy or if you're advocating allocating as much as possible up-front, but I'm curious if you have any thoughts on this: I always end up using (Rust) vectors despite looking at a bunch of slab/arena allocation libraries. Preferably I'd know how much memory I need up front, but barring that I see three…
[0] https://man7.org/linux/man-pages/man2/mremap.2.html
[1] https://git.musl-libc.org/cgit/musl/tree/src/malloc/mallocng...
But you need a better color scheme for your light theme my guy. Greys on greys on whites with bright pinks and light blues…it’s really hard to just read.
Earlier quoted context omitted.
That doesn't mean you should default to a slower implementation for new code. If you do, you're just creating more low-hanging fruit that nobody will find time to solve.
in many cases often its faster just to switch from debug to release - compilers are good to vectorise many loops. Worth to give it a try before rewriting clean loop/code into SIMD/NEON.
I like SIMD, but before super-optimizing your code with SIMD and the like, really consider your data structures and access patterns. I've been singing Data-Oriented Design's praises, so I'll just collect all my comments here [1], but I think it's a good approach to optimization. I played around with SIMD in my old code (in Zig), but my approach to modelling datastructures was so antithetical to optimization, it was l…
The nice part is that data-oriented code almost always easily supports threading and SIMD.
[flagged]