Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Removing characters from strings faster with AVX-512

lemire.me

Post reply on HN
Print view View original

61–70 of 88 posts

Re: Removing characters from strings faster with AVX-512

#61

Post by raphlinus »

fancyfredbot wrote: post #57
raphlinus wrote: post #29

Earlier quoted context omitted.

The short answer is no, but the long answer is that this is a very complex tradeoff space. Going forward, we may see more of these types of tasks moving to GPU, but for the moment it is generally not a good choice. The GPU is incredible at raw throughput, and this particular problem can actually implemented fairly straightforwardly (it's a stream compaction, which in turn can be expressed in terms of prefix sum). How…

I wonder if this applies to the same extent for an on-package GPU which shares the same physical memory as the CPU. I'd expect round trip times in that case to be minimal and the available processing power would probably be competitive with AVX512. I have been wondering if this is the reason for deprecating AVX512 on consumer processors - these are likely to have a GPU available.

Good question! There are two separate issues with putting the GPU in the same package as the CPU. One is the memcpy bandwidth issue, which is indeed entirely mitigated (assuming the app is smart enough to exploit this). But the round trip times seem more related to context switches. I have an M1 Max here, and just found ~200µs for a very simple dispatch (just clearing 16k of memory).

I personally believe it may be possible to reduce latency using techniques similar to io_uring, but it may not be simple. Likely a major reason for the roundtrips is so that a trusted process (part of the GPU driver) can validate inputs from untrusted user code before it's presented to the GPU hardware.

Re: Removing characters from strings faster with AVX-512

#62

Post by willis936 »

PragmaticPulp wrote: post #30
Andoryuuta wrote: post #22

Intel is removing AVX-512 support from their newer CPU's (Alder Lake +). :/ https://www.igorslab.de/en/intel-deactivated-avx-512-on-alde...

Server and workstation chips still have AVX-512. It’s only unsupported on CPUs with smaller E(fficeincy) cores. AVX-512 was never really supported in newer consumer CPUs with heterogeneous architecture. These CPUs have a mix of powerful cores and efficiency cores. The AVX-512 instructions were never added to the efficiency cores because it would use way too much die space and defeat the purpose of efficiency cores. T…

>The AVX-512 instructions were never added to the efficiency cores because it would use way too much die space and defeat the purpose of efficiency cores.

Isn't the purpose of efficiency cores to be more power efficient? It's more power efficient to vectorize instructions and minimize pipeline re-ordering.

Re: Removing characters from strings faster with AVX-512

#63

Post by tedunangst »

What would be a practical application of this? The linked post mentions a trim like operation, but in practice I only want to remove white space from the ends, not the interior of the string, and finding the ends is basically the whole problem. Or maybe I want to compress some json, but a simple approach won't work because there can be spaces inside string values which must be preserved.

Re: Removing characters from strings faster with AVX-512

#64

Post by mastax »

willis936 wrote: post #62
PragmaticPulp wrote: post #30

Earlier quoted context omitted.

Server and workstation chips still have AVX-512. It’s only unsupported on CPUs with smaller E(fficeincy) cores. AVX-512 was never really supported in newer consumer CPUs with heterogeneous architecture. These CPUs have a mix of powerful cores and efficiency cores. The AVX-512 instructions were never added to the efficiency cores because it would use way too much die space and defeat the purpose of efficiency cores. T…

>The AVX-512 instructions were never added to the efficiency cores because it would use way too much die space and defeat the purpose of efficiency cores. Isn't the purpose of efficiency cores to be more power efficient? It's more power efficient to vectorize instructions and minimize pipeline re-ordering.

Power and area efficient. You can fit 4 E cores in the area of 1 P core. Adding AVX-512 to the E cores would significantly hamper that, though I don't know by how much.

Re: Removing characters from strings faster with AVX-512

#65

Post by brrrrrm »

What's the generated assembly look like? I suspect clang isn't smart enough to store things into registers. The latency of VPCOMPRESSB seems quite high (according to the table here at least https://uops.info/table.html), so you'll probably want to induce a bit more pipelining by manually unrolling into the register variant.

I don't have an AVX512 machine with VBMI2, but here's what my untested code might look like:

  __m512i spaces = _mm512_set1_epi8(' ');
  size_t i = 0;
  for (; i + (64 * 4 - 1)
You can probably do better by chunking up the input and using temporary memory (coalesced at the end).

Re: Removing characters from strings faster with AVX-512

#66

Post by saagarjha »

pantalaimon wrote: post #39
PragmaticPulp wrote: post #30

Earlier quoted context omitted.

Server and workstation chips still have AVX-512. It’s only unsupported on CPUs with smaller E(fficeincy) cores. AVX-512 was never really supported in newer consumer CPUs with heterogeneous architecture. These CPUs have a mix of powerful cores and efficiency cores. The AVX-512 instructions were never added to the efficiency cores because it would use way too much die space and defeat the purpose of efficiency cores. T…

Catching the bad instruction fault on the E-cores and only scheduling the thread on the P-cores would be something that could be added to Linux (there were already third party patches towards that goal) if Intel had not disable the feature entirely.

Presumably the AVX-512 code is something on your hot path, so I’m not sure waiting for a signal to reschedule the work is something you would want.

Re: Removing characters from strings faster with AVX-512

#67

Post by R0b0t1 »

PragmaticPulp wrote: post #33
R0b0t1 wrote: post #31

Earlier quoted context omitted.

That's not a valid reason why I can't use them on the P cores. Some motherboards can enable them on the i9-12900k, it works fine, but you need to pin to a P core.

The reason is that it was never validated or tested with AVX-512 and Intel and motherboard vendors couldn’t commit to shipping everything with AVX-512 support in future steppings/revisions. If you disable E cores you could enable AVX-512 on certain motherboards, but like I said that’s not really a net win 99.99% of the time when you’re giving up entire cores. It was also at your own risk because presumably the power/…

Still smells like bullshit. Let the customer decide. Who cares if it was validated? Why was it even included? Just put it behind a yes-I-really-mean-it-switch so nobody uses it by accident.

Re: Removing characters from strings faster with AVX-512

#68

Post by zozbot234 »

PragmaticPulp wrote: post #30
Andoryuuta wrote: post #22

Intel is removing AVX-512 support from their newer CPU's (Alder Lake +). :/ https://www.igorslab.de/en/intel-deactivated-avx-512-on-alde...

Server and workstation chips still have AVX-512. It’s only unsupported on CPUs with smaller E(fficeincy) cores. AVX-512 was never really supported in newer consumer CPUs with heterogeneous architecture. These CPUs have a mix of powerful cores and efficiency cores. The AVX-512 instructions were never added to the efficiency cores because it would use way too much die space and defeat the purpose of efficiency cores. T…

> The AVX-512 instructions were never added to the efficiency cores because it would use way too much die space and defeat the purpose of efficiency cores.

And this is why scalable vector ISA's like the RISC-V vector extensions are superior to fixed-size SIMD. You can support both kinds of microarchitecture while running the exact same code.

Re: Removing characters from strings faster with AVX-512

#69

Post by jandrewrogers »

tedunangst wrote: post #63

What would be a practical application of this? The linked post mentions a trim like operation, but in practice I only want to remove white space from the ends, not the interior of the string, and finding the ends is basically the whole problem. Or maybe I want to compress some json, but a simple approach won't work because there can be spaces inside string values which must be preserved.

I agree that the whitespace in text example seems a bit contrived but I've done similar types of byte elision operations on binary streams (e.g. for compression purposes), which this could be trivially adapted to.

Re: Removing characters from strings faster with AVX-512

#70

Post by heavenlyblue »

saagarjha wrote: post #66
pantalaimon wrote: post #39

Earlier quoted context omitted.

Catching the bad instruction fault on the E-cores and only scheduling the thread on the P-cores would be something that could be added to Linux (there were already third party patches towards that goal) if Intel had not disable the feature entirely.

Presumably the AVX-512 code is something on your hot path, so I’m not sure waiting for a signal to reschedule the work is something you would want.

You reschedule it only once so it doesn’t matter
Post reply on HN