Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AVX-512, what’s useful for us

obe.tv

Post reply on HN
Print view View original

31–40 of 50 posts

Re: AVX-512, what’s useful for us

#31

Post by stagger87 »

raphlinus wrote: post #10

If I'm understanding it correctly, they're not actually using the 512 bit (ZMM) registers, because using them can cause overall system slowdown. It seems to me they're only really useful if you're doing an AVX-512 intensive workload. And do those really exist? For something like bulk matrix multiplications, GPGPU is going to be much better, both in throughput and in operations per joule. I'm remaining to be convinced…

Define significant. I write software that couldn't exist without SIMD, as do all my competitors. (Note: it could exist but wouldn't offer nearly the same level of capability, i.e. it would be a different product) By specifying the last several generations of Intel hardware I can guarantee customers can run my software (AVX1 minimum) with consumer level desktops/laptops. AVX-512 hardware support hasn't reached the average consumer yet, but I am ready to take advantage of it when it does. (signal/video processing)

Re: AVX-512, what’s useful for us

#32

Post by jabl »

dragontamer wrote: post #17
pjmlp wrote: post #12

Earlier quoted context omitted.

That is the beauty of JIT compilers, some JVMs like Azul's already support it.

Some of the AVX512 instruction set seems very well suited for automatic JIT compilers. I'm sure Azul immediately jumped on board for the "Conflict Detection" instruction set and are auto-vectorizing tons of more loops. But other AVX512 instructions don't seem very easy for compilers to automatically apply. In particular: Scatter/Gather instructions have implications on the most-efficient way to lay out data in memory…

AFAIK, scatter/gather have been available in supercomputer vector ISA's since the mid-1970'ies. And even with the state of compiler technology back then, the Cray Fortran compiler was able to use scatter/gather to vectorize loops with indirect addressing (e.g. a[ind[i]] ). Such as occurs e.g. in sparse matrix style computations.

Re: AVX-512, what’s useful for us

#33

Post by jabl »

dragontamer wrote: post #15
tambre wrote: post #9

Earlier quoted context omitted.

Couldn't you use something like GCC function multi-versioning?

AVX512 has so many more features above-and-beyond Intel's typical SIMD implementation. Feature wise, its beginning to be competitive against NVidia's PTX CUDA architecture. Like, AVX512 is a really, really good instruction set (or I guess: a really good set of instruction sets). Assuming AVX512 F, CD, VL, DQ, and BW (the expected AVX512 instructions in CannonLake): * AVX512F -- "Standard" 512-bit arithmetic already h…

I wonder why they did the BW thing instead of just defining a vector length register like other vector ISA's (which would have allowed to get rid of a remainder loop, leading to less code bloat and more efficient execution for short loops where the number of iterations is not an integer multiple of the ISA vector length).

Re: AVX-512, what’s useful for us

#34

Post by thecompilr »

dragontamer wrote: post #15
tambre wrote: post #9

Earlier quoted context omitted.

Couldn't you use something like GCC function multi-versioning?

AVX512 has so many more features above-and-beyond Intel's typical SIMD implementation. Feature wise, its beginning to be competitive against NVidia's PTX CUDA architecture. Like, AVX512 is a really, really good instruction set (or I guess: a really good set of instruction sets). Assuming AVX512 F, CD, VL, DQ, and BW (the expected AVX512 instructions in CannonLake): * AVX512F -- "Standard" 512-bit arithmetic already h…

VL - Extend AVX512 to operate on only 256-bit and 128-bits at a time. (vector length extension)

DQ - Extend AVX512 to Longs, Long Longs. (double word and quadword extension)

BW - Extend AVX512 to Bytes, Shorts. (byte and word extension)

Re: AVX-512, what’s useful for us

#35

Post by brigade »

justincormack wrote: post #28
Dylan16807 wrote: post #16

Earlier quoted context omitted.

What models don't slow down? Are you claiming the slowdown isn't actually necessary to keep the chip stable?

compare these https://en.wikichip.org/wiki/intel/xeon_silver/4116#Frequenc... https://en.wikichip.org/wiki/intel/xeon_gold/6154#Frequencie... https://en.wikichip.org/wiki/intel/xeon_platinum/8180m#Frequ... So platinum can run avx512 on 28 cores at 2.3GHz, while silver runs 12 cores at 1.4GHz

From your links, gold/plat has a ~15% clock speed hit when using ymm muls, then another ~20% hit when going to zmm muls.

So for 512-bit wide to not slow things down on Intel's chips, you need ~30% runtime of all cores to already be in AVX.

Re: AVX-512, what’s useful for us

#36

Post by steve_musk »

raphlinus wrote: post #10

If I'm understanding it correctly, they're not actually using the 512 bit (ZMM) registers, because using them can cause overall system slowdown. It seems to me they're only really useful if you're doing an AVX-512 intensive workload. And do those really exist? For something like bulk matrix multiplications, GPGPU is going to be much better, both in throughput and in operations per joule. I'm remaining to be convinced…

I work in an HPC lab (computational chemistry) and we have a hand-coded AVX-512 codepath. I can't give any specifics (because I don't know them) but I know there is a non-trivial speedup versus the standard codepath (or AVX2).

However, GPUs blow it out of the water for much a lower price since we only need FP32. I think the main reason we invested time adding supports is for the Xeon Phi cards. I guess is could be worthwhile for some FP64 pipelines from a cost/performance perspective.

Re: AVX-512, what’s useful for us

#37

Post by thecompilr »

pkaye wrote: post #25
mscrivo wrote: post #23

Earlier quoted context omitted.

AVX code is known to make the CPU run way hotter than usual. Perhaps that caused throttling that made general code running at the same time, or within a short span thereafter, perform worse?

That is one theory I had but i'm not sure how to determine if CPU is throttling (on Ubuntu Linux.)

You can either use lscpu, which is less accurate, or the best way is to check:

cat /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_cur_freq

where instead of cpu0 you can write any core number, and it will give you the current frequency of that core in KHz.

Re: AVX-512, what’s useful for us

#38

Post by mycall »

stagger87 wrote: post #31
raphlinus wrote: post #10

If I'm understanding it correctly, they're not actually using the 512 bit (ZMM) registers, because using them can cause overall system slowdown. It seems to me they're only really useful if you're doing an AVX-512 intensive workload. And do those really exist? For something like bulk matrix multiplications, GPGPU is going to be much better, both in throughput and in operations per joule. I'm remaining to be convinced…

Define significant. I write software that couldn't exist without SIMD, as do all my competitors. (Note: it could exist but wouldn't offer nearly the same level of capability, i.e. it would be a different product) By specifying the last several generations of Intel hardware I can guarantee customers can run my software (AVX1 minimum) with consumer level desktops/laptops. AVX-512 hardware support hasn't reached the ave…

Have you looked at RISC-V Vector proposal yet? Same person who did AVX-512 for Intel.

Re: AVX-512, what’s useful for us

#39

Post by pkaye »

thecompilr wrote: post #37
pkaye wrote: post #25

Earlier quoted context omitted.

That is one theory I had but i'm not sure how to determine if CPU is throttling (on Ubuntu Linux.)

You can either use lscpu, which is less accurate, or the best way is to check: cat /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_cur_freq where instead of cpu0 you can write any core number, and it will give you the current frequency of that core in KHz.

I'll try that out. Thanks.

Re: AVX-512, what’s useful for us

#40

Post by raphlinus »

stagger87 wrote: post #31
raphlinus wrote: post #10

If I'm understanding it correctly, they're not actually using the 512 bit (ZMM) registers, because using them can cause overall system slowdown. It seems to me they're only really useful if you're doing an AVX-512 intensive workload. And do those really exist? For something like bulk matrix multiplications, GPGPU is going to be much better, both in throughput and in operations per joule. I'm remaining to be convinced…

Define significant. I write software that couldn't exist without SIMD, as do all my competitors. (Note: it could exist but wouldn't offer nearly the same level of capability, i.e. it would be a different product) By specifying the last several generations of Intel hardware I can guarantee customers can run my software (AVX1 minimum) with consumer level desktops/laptops. AVX-512 hardware support hasn't reached the ave…

I don't want to overstate it; I've written a lot of SIMD in my life too. If you've got audio processing that needs to run in a realtime thread, I can see how SIMD is appealing, because we don't really have mechanisms to achieve realtime on GPU (yet). However, for a lot of seriously heavy computation, it seems like doing it on the GPU is a win. For the cases where memory transfer back and forth to the GPU is expensive, even integrated GPU should offer significantly more computational resources than even AVX-512.

One of the use cases I'm thinking of is font rendering, where I published a SIMD-heavy prototype a couple years ago, and it's blown out of the water by a newer GPU-based approach: http://pcwalton.github.io/blog/2017/02/14/pathfinder/

Post reply on HN