61–70 of 132 posts
I love how ever example of portable SIMD isn't portable. They specifies a constant SIMD width so it's non-portable. Well, not performance portable, but why are we using SIMD again?
Go's implementation is vector size independant https://pkg.go.dev/simd@master
Earlier quoted context omitted.
Why should it be portable? Honest question. SIMD seems to me, to be very platform specific. Maybe there are times one SIMD unit is not anothers' SIMD unit?
The create is called portable_simd. There is no reason a portable_simd relu_dot implemention should need to specify the SIMD width. But the design and documentation of portable_simd makes the fixed size syntactically easy/the default and the width agnostic code harder.
What should it choose then? I have a Zen 3 processor, and benchmarking some simd I did recently says 32 byte or 64 byte chunks was fastest. But I'm sure I'd get a different result on a different Zen, and different again on Intel's.
How would the library decide what SIMD width I should use?
`core` instead of `std` is great too!
This will become useful in one of my sideproject where I use bitmaps to speed up pathfinding, exited to try it out!
Earlier quoted context omitted.
The create is called portable_simd. There is no reason a portable_simd relu_dot implemention should need to specify the SIMD width. But the design and documentation of portable_simd makes the fixed size syntactically easy/the default and the width agnostic code harder.
> There is no reason a portable_simd relu_dot implemention should need to specify the SIMD width. What should it choose then? I have a Zen 3 processor, and benchmarking some simd I did recently says 32 byte or 64 byte chunks was fastest. But I'm sure I'd get a different result on a different Zen, and different again on Intel's. How would the library decide what SIMD width I should use?
Earlier quoted context omitted.
Go's implementation is vector size independant https://pkg.go.dev/simd@master
Sure but there's no real way to use that in a portable way, at least not a way that maximises performance on every CPU you run it on. That's pretty much impossible at the moment.
For many problems, choosing the right instruction or instruction sequence makes a large difference. Portable SIMD abstractions necessarily expose some common semantic layer, but SIMD ISAs don't actually have equivalent capabilities. Instructions like pshufb, for example, enable algorithmic tricks that don't necessarily have an equally efficient analogue on another architecture.
If maximum performance matters, I generally want intrinsics and architecture-specific implementations; if portability matters more, I'd rather move further up the abstraction stack and use something designed to target multiple architectures, such as ISPC. There are certainly cases where portable SIMD gets close enough to optimal, but I don't think there's a compiler or abstraction that can express every useful SIMD idiom and lower it equally efficiently across fundamentally different ISAs.
Earlier quoted context omitted.
> There is no reason a portable_simd relu_dot implemention should need to specify the SIMD width. What should it choose then? I have a Zen 3 processor, and benchmarking some simd I did recently says 32 byte or 64 byte chunks was fastest. But I'm sure I'd get a different result on a different Zen, and different again on Intel's. How would the library decide what SIMD width I should use?
It'd need some kind of compile-time hardware-feature-detection, yea? That seems probably feasible since proc macros can do essentially anything they like (worryingly).
What you'd actually want is a matrix of variant implementations burned into the binary, with runtime (or process-boot-time) hardware detection that swaps symbols out to point to the correct variant.
Earlier quoted context omitted.
I don't get what's the value of it not being enabled by default what does the toggle get us, really? Maybe I don't understand web design and it makes it harder to read for some, I am dyslexic and never had any issues.
It's just a way for us to add minutia and details that most don't care / need to know about. There are three audiences we try to make the posts accessible for: Rust people who don't know about GPUs, GPU people who don't know about Rust, and non-Rust non-GPU people. The toggle lets knowledgable readers go "wait, what about..." and hopefully the toggle answers it.
Author here, AMA.
If you have to express your computation using an "array programming DSL" with things like scan and gather anyways - why not opt to use torch/tensorflow/jax or anything else that targets MLIR? An example of writing a relu using an embedded array DSL is really not helping your case either - that's exactly the problem that these other solutions mentioned above are successfully solving for the past ~15y (starting with th…
Earlier quoted context omitted.
It's just a way for us to add minutia and details that most don't care / need to know about. There are three audiences we try to make the posts accessible for: Rust people who don't know about GPUs, GPU people who don't know about Rust, and non-Rust non-GPU people. The toggle lets knowledgable readers go "wait, what about..." and hopefully the toggle answers it.
Does making it the default hurt anyone though? I don't think that 1 subscript adds anything to it..
I would love to have an open source Rust SIMD library with the scope and maturity that https://github.com/google/highway brings to C++.