Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Removing characters from strings faster with AVX-512

lemire.me

Post reply on HN
Print view View original

51–60 of 88 posts

Re: Removing characters from strings faster with AVX-512

#51

Post by mwcampbell »

raphlinus wrote: post #29
protoman3000 wrote: post #20

Please correct me if I'm wrong, but wouldn't we normally scale these things instead on a GPU?

The short answer is no, but the long answer is that this is a very complex tradeoff space. Going forward, we may see more of these types of tasks moving to GPU, but for the moment it is generally not a good choice. The GPU is incredible at raw throughput, and this particular problem can actually implemented fairly straightforwardly (it's a stream compaction, which in turn can be expressed in terms of prefix sum). How…

> To give round numbers, the roundtrip latency is on the order of 100µs

I can't help but notice that, at least in my experience on Windows, this is the same order of magnitude as for inter-process communication on the local machine. Tangent: That latency was my nemesis as a Windows screen reader developer; the platform accessibility APIs weren't well designed to take it into account. Windows 11 finally has a good solution for this problem (yes, I helped implement that while I was at Microsoft).

Re: Removing characters from strings faster with AVX-512

#52

Post by janwas »

gslin wrote: post #11

A problem is slowing down the CPU frequency significantly when AVX-512 is involved, e.g. https://en.wikichip.org/wiki/intel/xeon_gold/6262v this, which usually cancels out the benefit in the Real World (tm).

I would love to see an example of reasonable code not seeing any benefit. On first generation SKX, we saw 1.5x speedups vs AVX2, and that was IIRC even without taking much advantage of AVX3-only instructions.

Re: Removing characters from strings faster with AVX-512

#53

Post by janwas »

adgjlsfhk1 wrote: post #46
PragmaticPulp wrote: post #36

Earlier quoted context omitted.

This was massively exaggerated by journalists when AVX-512 was first announced. It is true that randomly applied AVX-512 instructions can cause a slight clock speed reduction, the proper way to use libraries like this would be within specific hot code loops where the mild clock speed reduction is more than offset by the huge parallelism increase. This doesn’t make sense if you’re a consumer doing something multitaski…

the thing I never understood about this is why Intel didn't just add latency to the avx512 instructions instead? that seems much easier than downclocking the whole cpu

I believe they do actually do something like this - until power and voltage delivery change, wide instructions are throttled independently of frequency changes (which on SKX involved a short halt).

Re: Removing characters from strings faster with AVX-512

#54

Post by Twirrim »

mhh__ wrote: post #16
gslin wrote: post #11

A problem is slowing down the CPU frequency significantly when AVX-512 is involved, e.g. https://en.wikichip.org/wiki/intel/xeon_gold/6262v this, which usually cancels out the benefit in the Real World (tm).

Unless someone has data for the latest Intel chips (i.e. sapphire rapids) showing the opposite I'm inclined to think this is a meme from 2016/7 that needs to go the way of the dodo.

It was largely wrong then, too. Cloudflare, who really kicked off a large amount of the fuss, had "Bronze" class Xeon chips, that weren't designed or marketed for what they were attempting to use them for. They were only ever intended for small business stuff. Not large scale high performance operations. Their performance downclock for AVX-512 is way, way higher on Bronze.

Re: Removing characters from strings faster with AVX-512

#55

Post by NavinF »

Twirrim wrote: post #54
mhh__ wrote: post #16

Earlier quoted context omitted.

Unless someone has data for the latest Intel chips (i.e. sapphire rapids) showing the opposite I'm inclined to think this is a meme from 2016/7 that needs to go the way of the dodo.

It was largely wrong then, too. Cloudflare, who really kicked off a large amount of the fuss, had "Bronze" class Xeon chips, that weren't designed or marketed for what they were attempting to use them for. They were only ever intended for small business stuff. Not large scale high performance operations. Their performance downclock for AVX-512 is way, way higher on Bronze.

Weren’t those chips $10k each back then? Hardly anyone got gold Xeons.

Re: Removing characters from strings faster with AVX-512

#56

Post by pclmulqdq »

adrian_b wrote: post #47
pclmulqdq wrote: post #34

Earlier quoted context omitted.

AVX-512 finally gets a lot of things right about vector manipulation and plugged a lot of the holes in the instruction set. Part of me is upset that it came with the "512" name - they could have called it "AVX3" or "AVX Version 2" (since it's intel and they love confusing names).

Actually AVX-512 predates AVX and Sandy Bridge. The original name of AVX-512 was "Larrabee New Instructions". Unlike with the other Intel instruction set extensions, the team which defined the "Larrabee New Instructions" included graphics experts hired from outside Intel, which is probably the reason why AVX-512 is a better SIMD instruction set than all the other designed by Intel. Unfortunately, Sandy Bridge (2011),…

TIL. Thank you for the history lesson on AVX. Comparing to SVE and the RISC-V vector instructions, AVX feels so clunky, but I guess that was part of the "Intel tax."

Re: Removing characters from strings faster with AVX-512

#57

Post by fancyfredbot »

raphlinus wrote: post #29
protoman3000 wrote: post #20

Please correct me if I'm wrong, but wouldn't we normally scale these things instead on a GPU?

The short answer is no, but the long answer is that this is a very complex tradeoff space. Going forward, we may see more of these types of tasks moving to GPU, but for the moment it is generally not a good choice. The GPU is incredible at raw throughput, and this particular problem can actually implemented fairly straightforwardly (it's a stream compaction, which in turn can be expressed in terms of prefix sum). How…

I wonder if this applies to the same extent for an on-package GPU which shares the same physical memory as the CPU. I'd expect round trip times in that case to be minimal and the available processing power would probably be competitive with AVX512. I have been wondering if this is the reason for deprecating AVX512 on consumer processors - these are likely to have a GPU available.

Re: Removing characters from strings faster with AVX-512

#58

Post by [deleted] »

37ef_ced3 wrote: post #9
mdb31 wrote: post #2

Cool performance enhancement, with an accompanying implementation in a real-world library ( https://github.com/lemire/despacer ). Still, what does it signal that vector extensions are required to get better string performance on x86? Wouldn't it be better if Intel invested their AVX transistor budget into simply making existing REPB prefixes a lot faster?

AVX-512 is an elegant, powerful, flexible set of masked vector instructions that is useful for many purposes. For example, low-cost neural net inference ( https://NN-512.com ). To suggest that Intel and AMD should instead make "existing REPB prefixes a lot faster" is missing the big picture. The masked compression instructions (one of which is used in Lemire's article) are endlessly useful, not just for stripping spa…

[deleted]

Re: Removing characters from strings faster with AVX-512

#59

Post by bertr4nd »

I love Daniel’s vectorized string processing posts. There’s always some clever trickery that’s hard for a guy like me (who mostly uses vector extensions for ML kernels) to get quickly.

I found myself wondering if one could create a domain-specific language for specifying string processing tasks, and then automate some of the tricks with a compiler (possibly with human-specified optimization annotations). Halide did this sort of thing for image processing (and ML via TVM to some extent) and it was a pretty significant success.

Re: Removing characters from strings faster with AVX-512

#60

Post by Twirrim »

NavinF wrote: post #55
Twirrim wrote: post #54

Earlier quoted context omitted.

It was largely wrong then, too. Cloudflare, who really kicked off a large amount of the fuss, had "Bronze" class Xeon chips, that weren't designed or marketed for what they were attempting to use them for. They were only ever intended for small business stuff. Not large scale high performance operations. Their performance downclock for AVX-512 is way, way higher on Bronze.

Weren’t those chips $10k each back then? Hardly anyone got gold Xeons.

Not even close. The blog post was 2017.

Actually, I stand corrected, after double checking, Cloudflare were using Silver. Entry level data centre chips, instead of small business chips. Still not the kind of chips you'd buy for high performance infrastructure, and not intended to be used for such.

Xeon Silver 4116s hit the market at $1,002.00. The Golds were $1,221.00. The performance differences are quite significant. For something that'll be in service for ~3-5 years, $200 is absolutely trivial by way of a per-chip increase. It's firmly in the "false economy" territory to be skimping on your chip costs. It's a bit more understandable in smaller businesses, but you just don't do it when you're operating at scale.

Also remember: at the scales that Cloudflare are purchasing at, they won't be paying RRP. They'll be getting tidy discounts.

Post reply on HN