Removing characters from strings faster with AVX-512
1–10 of 88 posts
Re: Removing characters from strings faster with AVX-512
#2Cool performance enhancement, with an accompanying implementation in a real-world library (https://github.com/lemire/despacer).
Still, what does it signal that vector extensions are required to get better string performance on x86? Wouldn't it be better if Intel invested their AVX transistor budget into simply making existing REPB prefixes a lot faster?
Re: Removing characters from strings faster with AVX-512
#3Cool performance enhancement, with an accompanying implementation in a real-world library ( https://github.com/lemire/despacer ). Still, what does it signal that vector extensions are required to get better string performance on x86? Wouldn't it be better if Intel invested their AVX transistor budget into simply making existing REPB prefixes a lot faster?
Why is a large speedup from vectors surprising? Considering that the energy required for scheduling/dispatching an instruction on OoO cores dwarfs that of the actual operation (add/mul etc), amortizing over multiple elements (=SIMD) is an obvious win.
Re: Removing characters from strings faster with AVX-512
#4Cool performance enhancement, with an accompanying implementation in a real-world library ( https://github.com/lemire/despacer ). Still, what does it signal that vector extensions are required to get better string performance on x86? Wouldn't it be better if Intel invested their AVX transistor budget into simply making existing REPB prefixes a lot faster?
Why is a large speedup from vectors surprising? Considering that the energy required for scheduling/dispatching an instruction on OoO cores dwarfs that of the actual operation (add/mul etc), amortizing over multiple elements (=SIMD) is an obvious win.
Where do I say that the speedup is surprising?
My question is whether Intel investing in AVX-512 is wise, given that: -Most existing code is not aware of AVX anyway; -Developers are especially wary of AVX-512, since they expect it to be discontinued soon.
Consequently, wouldn't Intel be better off by using the silicon dedicated to AVX-512 to speed up instruction patterns that are actually used?
Re: Removing characters from strings faster with AVX-512
#5Earlier quoted context omitted.
Why is a large speedup from vectors surprising? Considering that the energy required for scheduling/dispatching an instruction on OoO cores dwarfs that of the actual operation (add/mul etc), amortizing over multiple elements (=SIMD) is an obvious win.
Where do I say that the speedup is surprising? My question is whether Intel investing in AVX-512 is wise, given that: -Most existing code is not aware of AVX anyway; -Developers are especially wary of AVX-512, since they expect it to be discontinued soon. Consequently, wouldn't Intel be better off by using the silicon dedicated to AVX-512 to speed up instruction patterns that are actually used?
My point is that vector instructions are fundamentally necessary and thus "what does it signal" evaluates to "nothing surprising".
Sure, REP STOSB/MOVSB make for a very compact memset/memcpy, but their performance varies depending on CPU feature flags, so you're going to want multiple codepaths anyway. And vector instructions are vastly more flexible than just those two.
Also, I have not met developers who expect AVX-512 to be discontinued (the regrettable ADL situation notwithstanding; that's not a server CPU). AMD is actually adding AVX-512.
Re: Removing characters from strings faster with AVX-512
#6Earlier quoted context omitted.
Where do I say that the speedup is surprising? My question is whether Intel investing in AVX-512 is wise, given that: -Most existing code is not aware of AVX anyway; -Developers are especially wary of AVX-512, since they expect it to be discontinued soon. Consequently, wouldn't Intel be better off by using the silicon dedicated to AVX-512 to speed up instruction patterns that are actually used?
My point is that vector instructions are fundamentally necessary and thus "what does it signal" evaluates to "nothing surprising". Sure, REP STOSB/MOVSB make for a very compact memset/memcpy, but their performance varies depending on CPU feature flags, so you're going to want multiple codepaths anyway. And vector instructions are vastly more flexible than just those two. Also, I have not met developers who expect AVX…
> vector instructions are fundamentally necessary
For which percentage of users?
> AMD is actually adding AVX-512
Which is irrelevant to in-market support for that instruction set.
Re: Removing characters from strings faster with AVX-512
#7Earlier quoted context omitted.
My point is that vector instructions are fundamentally necessary and thus "what does it signal" evaluates to "nothing surprising". Sure, REP STOSB/MOVSB make for a very compact memset/memcpy, but their performance varies depending on CPU feature flags, so you're going to want multiple codepaths anyway. And vector instructions are vastly more flexible than just those two. Also, I have not met developers who expect AVX…
> vector instructions are fundamentally necessary For which percentage of users? > AMD is actually adding AVX-512 Which is irrelevant to in-market support for that instruction set.
Why would it be irrelevant? Even the paucity of availability isn't really a problem - the big winners here are server users in data centers, not desktops or laptops. How much string parsing and munging is happening ingesting big datasets right now? If running a specially optimized function set on part of your fleet reduces utilization, that's direct cost savings you realize. If the AMD is then widening that support base, you're deeply favoring expanding usage while you scale up.
Re: Removing characters from strings faster with AVX-512
#8Earlier quoted context omitted.
> vector instructions are fundamentally necessary For which percentage of users? > AMD is actually adding AVX-512 Which is irrelevant to in-market support for that instruction set.
Why would it be irrelevant? Even the paucity of availability isn't really a problem - the big winners here are server users in data centers, not desktops or laptops. How much string parsing and munging is happening ingesting big datasets right now? If running a specially optimized function set on part of your fleet reduces utilization, that's direct cost savings you realize. If the AMD is then widening that support b…
Given Intel's AVX extension could cause silent failures on servers (very high work load for prolonged time, compare to end user computers), I'm not sure it would be a big win for servers either: https://arxiv.org/pdf/2102.11245.pdf.
Re: Removing characters from strings faster with AVX-512
#9Cool performance enhancement, with an accompanying implementation in a real-world library ( https://github.com/lemire/despacer ). Still, what does it signal that vector extensions are required to get better string performance on x86? Wouldn't it be better if Intel invested their AVX transistor budget into simply making existing REPB prefixes a lot faster?
AVX-512 is an elegant, powerful, flexible set of masked vector instructions that is useful for many purposes. For example, low-cost neural net inference (https://NN-512.com). To suggest that Intel and AMD should instead make "existing REPB prefixes a lot faster" is missing the big picture. The masked compression instructions (one of which is used in Lemire's article) are endlessly useful, not just for stripping spaces out of a string!
Re: Removing characters from strings faster with AVX-512
#10Earlier quoted context omitted.
My point is that vector instructions are fundamentally necessary and thus "what does it signal" evaluates to "nothing surprising". Sure, REP STOSB/MOVSB make for a very compact memset/memcpy, but their performance varies depending on CPU feature flags, so you're going to want multiple codepaths anyway. And vector instructions are vastly more flexible than just those two. Also, I have not met developers who expect AVX…
> vector instructions are fundamentally necessary For which percentage of users? > AMD is actually adding AVX-512 Which is irrelevant to in-market support for that instruction set.
Any users who either wants performance or uses a language that can depend on a fast library.