221–230 of 263 posts
If you’re processing N things at a time and your “scalar tail” is N-1 why can’t you put in a dummy value for the last entry, run one last SIMD iteration, and discard the dummy return value?
Earlier quoted context omitted.
> best effort parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU with a small directive. I doubt GPU is included by most runtimes yet, but for the rest of that have you tried SQL?
Good answer, too many folks miss out how powerful SQL actually is, and with stored procedures its compilation to native code can even cached across executions.
The big problem with databases, in my opinion, is the horrible API friction between them and your code (not even SQL per se). It makes sense if you're calling out to a database server and transferring data, but for the small, intermediate values we see in code every day, the relational model is amazing and yet so painful to use within a given programming language.
I've been envying the C# people and their LINQ, and the Java people and their jOOQ, because I'm either making a half-assed database in my own code with structs, arrays, and hashmaps, or I'm constructing some SQL monstrosity, shoveling it out to SQLite or DuckDB through a library, and marshalling the types back and forth.
Why can't I just have everything I want all the time?
Earlier quoted context omitted.
Your compiler can vectorise trivial things, once your code gets complicated it will no longer vectorise.
Once the code is complicated you don't need to make it more complicated lest it becomes unmaintainable.
Earlier quoted context omitted.
Case-in-point, the example in my own post doesn't auto-vectorize with LLVM or GCC at highest optimization levels. Basically, compilers will never auto-vectorize loops with an early loop break afaik.
You need to let the compiler know that there are at least 4 or 8 elements to process. This may require padding data and/or having a second loop after the main one that processes the remainder You start the post with: > There is an opportunity to use SIMD. SIMD turns those into this: > > for (8 byte chunk in bytes) { /* ... */ } If you actually wrote that loop, there is a good chance the compiler (gcc specifically) wi…
Counterpoint: https://pharr.org/matt/blog/2018/04/18/ispc-origins
> I think that the fatal flaw with the approach the compiler team was trying to make work was best diagnosed by T. Foley, who’s full of great insights about this stuff: auto-vectorization is not a programming model.
> The problem with an auto-vectorizer is that as long as vectorization can fail (and it will), then if you’re a programmer who actually cares about what code the compiler generates for your program, you must come to deeply understand the auto-vectorizer. Then, when it fails to vectorize code you want to be vectorized, you can either poke it in the right ways or change your program in the right ways so that it works for you again. This is a horrible way to program; it’s all alchemy and guesswork and you need to become deeply specialized about the nuances of a single compiler’s implementation—something you wouldn’t otherwise need to care about one bit.
> And God help you when they release a new version of the compiler with changes to the auto-vectorizer’s implementation.
> With a proper programming model, then the programmer learns the model (which is hopefully fairly clean), one or more compilers implement it, the generated code is predictable (no performance cliffs), and everyone’s happy.
I just do gcc -O3 and get SIMD without having to learn it
I have no idea why you're being downvoted. HN has a fetish for SIMD, but if you are hand-rolling SIMD and you aren't writing an explicit acceleration library, you're doing it wrong. Like, 100% of the time. Every modern language has a vectorization optimizing compiler, and through some fairly straightforward techniques this is automagic. And contrary to the various replies, unless you screwed something up compilers ar…
What if the “explicit acceleration library” for what you need to do doesn’t exist?
It distresses me that we don’t have a language that can do a best effort parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU with a small directive. I don’t need it to be optimal, just … handy as an option! The last time I brought this up here, folks offered a bunch of options that don’t quite do this, and the best candidate was this 15 year old compiler project that is I…
>It distresses me that we don’t have a language that can do a best effort parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU with a small directive. It's called ParaSail. The next closest thing to ParaSail is ... ... literally just Rust. Why? Because ParaSail has completely eliminated pointers, thereby preventing pointer aliasing. It can't be understated that pointer ali…
> All of the objects declared in a given scope are associated with a storage region, essentially a local heap. As an object grows, all new storage for it is allocated out of this region. As an object shrinks, the old storage can be immediately released back to this region. When a scope is exited, the entire region is reclaimed. There is no need for asynchronous garbage collection, as garbage never accumulates. Objects may grow in a highly irregular fashion without losing their locality of reference.
> Note that pointers are still used behind the scenes in the ParaSail implementation, but eliminating them from the surface syntax and semantics eliminates the complexity associated with pointers.
This approach seems to come up in many contexts, where a pointer-based address (raw pointer, reference, slice, etc.) is abstracted into a higher-level address key, usually an index integer (essentially a higher-level virtual pointer). The implementation might reallocate under the surface, or manage chunks of data through some sort of paging where the underlying pointers don't change (I was musing about this here [2]).
I guess I'm thinking out loud here, but most dynamic languages (eg. Python) don't expose the pointers or care about invalidation of the addresses, they just happily reallocate. I've barely written parallelized code, is pointer aliasing really one of the biggest roadblocks? It seems like it can be abstracted away fairly easily, even in a pointer-exposing language.
1. https://www.adacore.com/uploads/papers/parasail-pointer-free...
Earlier quoted context omitted.
I had the same thoughts about SIMD code being too verbose when I wrote some, so a few months ago I tried writing a library that lets you write quasi-GLSL code in C++, so much more compact, with the ability to switch between SIMD width without having to rewrite anything at all: https://github.com/gitdepierre/cppshader Not sure it will ever be useful, but it was a fun pet project with some interesting problems to solve…
Is GLSL more "approachable" than SIMD? For me personally (who doesn't have any graphics programming experience), GLSL feels way scarier than SIMD, especially when you look at the black magic that happens on shadertoys. Not saying GLSL is actually hard, but for a programmer like me SIMD might actually be more approachable
I got the idea to write the library when I wanted a simplex noise function in C++, implemented with AVX2 for performance reasons (because why not). There were a lot of public HLSL/GLSL implementations that were concise and fast on GPU, some of which I had used for years for my own needs, but rewriting the whole code in plain C++ would have been a hassle, and i would have to do the same for every GPU code i came accross in the future, so building a wrapper seemed like the most efficient approach.
So in the end, the library is mostly aimed at people coming from the GPU world who want to keep most of their habits. But yeah, there is probably very few use cases for it.
we make extensive use of simd at work, usually through highway: https://github.com/google/highway imo this is one of the greatest libs ever written. it handles dynamic dispatching of correct simd instructions / lane widths for various hardware with just one simd loop written (handling NEON/AVX/AVX2/AVX512/extensions) with comparable performance to handwritten native intrinsics
No, not really. Highway generates rather bad code as soon as you step outside a pretty narrow vertically-oriented scope, in my experience.
Earlier quoted context omitted.
I have no idea why you're being downvoted. HN has a fetish for SIMD, but if you are hand-rolling SIMD and you aren't writing an explicit acceleration library, you're doing it wrong. Like, 100% of the time. Every modern language has a vectorization optimizing compiler, and through some fairly straightforward techniques this is automagic. And contrary to the various replies, unless you screwed something up compilers ar…
> HN has a fetish for SIMD, but if you are hand-rolling SIMD and you aren't writing an explicit acceleration library, you're doing it wrong. Like, 100% of the time. What if the “explicit acceleration library” for what you need to do doesn’t exist?