Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Removing characters from strings faster with AVX-512

lemire.me

Post reply on HN
Print view View original

71–80 of 88 posts

Re: Removing characters from strings faster with AVX-512

#71

Post by fancyfredbot »

raphlinus wrote: post #61
fancyfredbot wrote: post #57

Earlier quoted context omitted.

I wonder if this applies to the same extent for an on-package GPU which shares the same physical memory as the CPU. I'd expect round trip times in that case to be minimal and the available processing power would probably be competitive with AVX512. I have been wondering if this is the reason for deprecating AVX512 on consumer processors - these are likely to have a GPU available.

Good question! There are two separate issues with putting the GPU in the same package as the CPU. One is the memcpy bandwidth issue, which is indeed entirely mitigated (assuming the app is smart enough to exploit this). But the round trip times seem more related to context switches. I have an M1 Max here, and just found ~200µs for a very simple dispatch (just clearing 16k of memory). I personally believe it may be po…

Yes I think you are right about driver overhead, although there should be ways to amortize that it probably doesn't work very well for latency sensitive problems! I expect that in most cases if you have enough work to do to make using AVX512 worthwhile you can afford the round-trip.

Re: Removing characters from strings faster with AVX-512

#72

Post by saagarjha »

heavenlyblue wrote: post #70
saagarjha wrote: post #66

Earlier quoted context omitted.

Presumably the AVX-512 code is something on your hot path, so I’m not sure waiting for a signal to reschedule the work is something you would want.

You reschedule it only once so it doesn’t matter

I don’t see how? Once your quantum is up the thread gets put back into the scheduling pool and you have to do this all over again…

Re: Removing characters from strings faster with AVX-512

#73

Post by nwmcsween »

ip26 wrote: post #44
mdb31 wrote: post #2

Cool performance enhancement, with an accompanying implementation in a real-world library ( https://github.com/lemire/despacer ). Still, what does it signal that vector extensions are required to get better string performance on x86? Wouldn't it be better if Intel invested their AVX transistor budget into simply making existing REPB prefixes a lot faster?

Is it generally possible to convert rep str sequences to AVX? Could the hardware or compiler already be doing this? AVX is just the SIMD unit. I would argue the transistors were spent on SIMD, and the hitch is simply the best way to send str commands to the SIMD hardware.

Why? IIRC something like 99% of string operations are on 20 chars or less. If you're hitting bottlenecks then optimize.

Re: Removing characters from strings faster with AVX-512

#74

Post by kadoban »

R0b0t1 wrote: post #67
PragmaticPulp wrote: post #33

Earlier quoted context omitted.

The reason is that it was never validated or tested with AVX-512 and Intel and motherboard vendors couldn’t commit to shipping everything with AVX-512 support in future steppings/revisions. If you disable E cores you could enable AVX-512 on certain motherboards, but like I said that’s not really a net win 99.99% of the time when you’re giving up entire cores. It was also at your own risk because presumably the power/…

Still smells like bullshit. Let the customer decide. Who cares if it was validated? Why was it even included? Just put it behind a yes-I-really-mean-it-switch so nobody uses it by accident.

There's not many great places for such a switch, and even then, then you have to validate _that_ behavior. It's mostly just not worth it.

Re: Removing characters from strings faster with AVX-512

#75

Post by boardwaalk »

saagarjha wrote: post #72
heavenlyblue wrote: post #70

Earlier quoted context omitted.

You reschedule it only once so it doesn’t matter

I don’t see how? Once your quantum is up the thread gets put back into the scheduling pool and you have to do this all over again…

Couldn’t there be some flag on the thread that marks it as “P-core only”? Doesn’t seem hard. I don’t known Linux scheduler internals though.

Re: Removing characters from strings faster with AVX-512

#76

Post by boulos »

raphlinus wrote: post #61
fancyfredbot wrote: post #57

Earlier quoted context omitted.

I wonder if this applies to the same extent for an on-package GPU which shares the same physical memory as the CPU. I'd expect round trip times in that case to be minimal and the available processing power would probably be competitive with AVX512. I have been wondering if this is the reason for deprecating AVX512 on consumer processors - these are likely to have a GPU available.

Good question! There are two separate issues with putting the GPU in the same package as the CPU. One is the memcpy bandwidth issue, which is indeed entirely mitigated (assuming the app is smart enough to exploit this). But the round trip times seem more related to context switches. I have an M1 Max here, and just found ~200µs for a very simple dispatch (just clearing 16k of memory). I personally believe it may be po…

It's been a while, but IIRC the integrated GPUs are only L3-cache coherent. So while that greatly improves the memcpy problem, anything that would have fit in L1 and does a bunch of math may still be a better fit for AVX2 or AVX-512.

Re: Removing characters from strings faster with AVX-512

#77

Post by boulos »

adrian_b wrote: post #47
pclmulqdq wrote: post #34

Earlier quoted context omitted.

AVX-512 finally gets a lot of things right about vector manipulation and plugged a lot of the holes in the instruction set. Part of me is upset that it came with the "512" name - they could have called it "AVX3" or "AVX Version 2" (since it's intel and they love confusing names).

Actually AVX-512 predates AVX and Sandy Bridge. The original name of AVX-512 was "Larrabee New Instructions". Unlike with the other Intel instruction set extensions, the team which defined the "Larrabee New Instructions" included graphics experts hired from outside Intel, which is probably the reason why AVX-512 is a better SIMD instruction set than all the other designed by Intel. Unfortunately, Sandy Bridge (2011),…

Hmm. That's not how I recall it. The folks in Israel working on Sandybridge (Gesher), already had their AVX plans in place before LRBni was "finalized" (even by the time of "our" siggraph paper -- I was only tangentially involved, not listed -- new instructions were being added all the time).

So it's more like both groups knew what the other was doing, but LRBni was free to focus primarily on graphics and a clean slate, while the AVX folks shot for "SSE but wider, and a few more".

AVX-512 is sort of a franken-combo of what AVX3 would have been, plus many of the LRBni instructions that shipped in the poorly named MIC parts, plus some more (e.g., now including a VNNI dialect, bf16 ops, etc.).

Re: Removing characters from strings faster with AVX-512

#78

Post by Dylan16807 »

PragmaticPulp wrote: post #33
R0b0t1 wrote: post #31

Earlier quoted context omitted.

That's not a valid reason why I can't use them on the P cores. Some motherboards can enable them on the i9-12900k, it works fine, but you need to pin to a P core.

The reason is that it was never validated or tested with AVX-512 and Intel and motherboard vendors couldn’t commit to shipping everything with AVX-512 support in future steppings/revisions. If you disable E cores you could enable AVX-512 on certain motherboards, but like I said that’s not really a net win 99.99% of the time when you’re giving up entire cores. It was also at your own risk because presumably the power/…

> it was never validated or tested with AVX-512 and Intel and motherboard vendors couldn’t commit

Only because they screwed it up on purpose! That's not an acceptable reason for removing the feature; in part because it would apply to any feature they decided to cut.

Re: Removing characters from strings faster with AVX-512

#79

Post by saagarjha »

boardwaalk wrote: post #75
saagarjha wrote: post #72

Earlier quoted context omitted.

I don’t see how? Once your quantum is up the thread gets put back into the scheduling pool and you have to do this all over again…

Couldn’t there be some flag on the thread that marks it as “P-core only”? Doesn’t seem hard. I don’t known Linux scheduler internals though.

Right but then you’d just have all processes with that flag all the time.

Re: Removing characters from strings faster with AVX-512

#80

Post by pantalaimon »

boardwaalk wrote: post #75
saagarjha wrote: post #72

Earlier quoted context omitted.

I don’t see how? Once your quantum is up the thread gets put back into the scheduling pool and you have to do this all over again…

Couldn’t there be some flag on the thread that marks it as “P-core only”? Doesn’t seem hard. I don’t known Linux scheduler internals though.

Threads already have an affinity bit mask where you can select which cores they can be scheduled on.

https://man7.org/linux/man-pages/man2/sched_setaffinity.2.ht...

Post reply on HN