> while with stuff like AVX there is this implicit hope that you just code as always and the compiler will take care of the rest via auto-vectorization and PhD level optimization algorithms No. I recently could really, really have used the packed saturated integer arithmetic and horizontal addition in AVX2 (but my old machine doesn't support it) and even better, the same but 512 bits wide on AVX512. It would only hav…
I am looking at it of the point of view of joe/jane developer that cannot tell head from tail regarding vector programming and doesn't even know what compiler intrinsics are for, and use languages that don't expose them anyway.
Well those people will never be getting the most out of their CPUs to begin with.
FP matters (especially with SIMD) It matters to image/video/audio processing It matters to simulations It matters to 3D models/rendering It matters to games So it's not "just benchmarks", people actually want to do stuff with it Sure, AVX512 might not be the greatest way of doing it, and it might be better to just make the existing instructions go faster, that might work
It's a matter of perspective. Back in the day, CPUs didn't come with FPUs and the latter were optional co-processors. The idea in the x86-world always was to "outsource" special requirements to dedicated hardware (FP co-processors, GPUs, sound cards, network cards, hardware codec cards, etc.), instead of putting them on the CPU package (like ARM-based SoCs). So it's different philosophies entirely - tightly integrate…
> The idea in the x86-world always was to "outsource" special requirements to dedicated hardware
Actually the Atari and Amigas were there first, that was PC catching up with their multimedia capabilities.
If he was right with FP, he'd know better than the business analysts at Intel. Instead, his opinion is based on what the market looked like thirty years ago. Nine years ago, AMD tested the hypothesis that really more "cores" and higher integer throughput were all that was needed and that FP performance didn't matter. The resulting architecture (Bulldozer) was a near-fatal disaster. It didn't even work out in the data…
AMD is currently giving intel great pain. So much for business analysts at Intel.
The only pain I see is they having won the CPU for game consoles.
All our laptops have Intel stickers on them and I doubt AMD is winning crazy dollars on cloud deployments.
The AVX512 instructions can cause strange global performance downgrades. “One challenge with AVX-512 is that it can actually _slow down_ your code. It's so power hungry that if you're using it on more than one core it almost immediately incurs significant throttling. Now, if everything you're doing is 512 bits at a time, you're still winning. But if you're interleaving scalar and vector arithmetic, the drop in clock…
wonder if this could be used as a denial of service against a vps host node.
He has the history correct. Most of the CPUs that x86 beat in the market had superior FP performance; SPARCs, Alphas, PA-RISC, Itanium, etc. This isn't a discussion about history, it's a discussion about SIMD FP instruction sets. Maybe we should all be astronomers and thermodynamicists writing bespoke finite element simulations and have a deep appreciation for the wonders of floating point ISAs, but that's just not t…
This discussion is about avx512, which has shown to have some issues compared to other solutions. Nobody is claiming FP is garbage or that we don't ever need SIMD. Regarding Linus: he is almost always right. He has had more hands-on experience than everyone in this thread combined. I trust his judgement, he has earned it by being consistently correct while his opposition has just complained and eventually failed to d…
He likely has ~zero hands on experience with SIMD or AVX512 as those aren't used in his precious kernel.
> The AVX512 instructions can cause strange global performance downgrades. Can other SIMD instructions (AVX2, say) do the same?
> Can other SIMD instructions (AVX2, say) do the same? On Intel CPUs, yes. There's even a BIOS/UEFI setting to specify how much you want the clock frequency to drop when running AVX code called "AVX offset". AMD CPUs doesn't do that though as far as I know. The thermal hit of using wider vectors decreases with every node shrink though, so expect the issue to become muted over time (which also explains why that doesn'…
AVX offset is only available on motherboards that support overclocking due to how much higher intel CPUs can be pushed relatively to their advertised base and boost clocks.
Both Zen and Intel lower their clocks under load especially AVX, keep in mind that Zen 2 doesn’t even reach its advertised boost clocks under any load some CPUs come close to within 100mhz or so but overall they all clock down rather fast once TMax or PMax is reached.
What are the forces in chip design that are at play here? Over the last 10-15 years, fabs have continued to fit more and more logic gates per unit area, but haven't reduced the power consumption per gate as much. As a result, if you fill your modern chip with compute gates, you cannot use them all at once because the chip will melt. Or at least you can't have them all running at max clock rates. One solution is to in…
Jim Keller had an interesting talk recently [1] about ways of doing parallel processing to better us the billions of transistors we have - assuming the task is parallelizable. There's the scalar core (i.e the basic CPU) which is easy to program realtively. Then a scalar core with vector instructions - difficult to program efficiently. Then there are arrays of scalar cores, i.e. GPUs, so relatively easy to program aga…
At least part of the problem is that computing mostly depends on moving data. Memory bandwidth is relatively low, so it's difficult to get enough actual floating point intensity, at least for "large" arrays even when it's theoretically available. A classic example is GEMM (generalized matrix multiplication) where you should expect a good implementation to get around 90% of peak performance, but also expect it to jump through various tricky hoops to get there. With, say, vector multiplication the hoops aren't available, and you're ultimately memory-bound.
Yes, there's more to it than that, and SIMD has non-FP applications etc.
None of those are both portable and widely available on end user machines, which is needed for games CUDA seems nice, but being Nvidia only makes it a total dead end.
Disclaimer: I work on AMD ROCm, but my opinions are my own. There's also HIP[1], which can be used as a thin wrapper around CUDA, or with the ROCm backend on AMD platforms. It doesn't yet match CUDA in either breadth of features or maturity, but it's getting closer every day. [1]: https://github.com/ROCm-Developer-Tools/HIP
As I understand it, that has to work for the CORAL 2 US "exascale", so people who've been proved fairly right so far obviously have some confidence in it. (de Supinksi of Livermore said he'd be out of a job if conventional wisdom was right, though it was pretty obvious at the time that it wasn't.)
Free software too, praise be.
The main problem is software, with GPGPUs you need to explicitly program for them, while with stuff like AVX there is this implicit hope that you just code as always and the compiler will take care of the rest via auto-vectorization and PhD level optimization algorithms. Because outside artificial intelligence, graphics and audio, there is little else that common applications would use the GPGPU for, so the large maj…
In my ideal world you'd be able to mark a function "this should compile to / run on gpgpu" and the compiler would potentially tell you why it can't do that. I'm not even sure if anything is stopping us apart from implementing that apart from the effort required. Sure, many ways to write that code will result in terrible performance, but it would still be closer to the auto-vectorisation experience. Actually we alread…
The current OpenMP spec has GPU offload features specifically for what was expected of the Sierra supercomputer. I'm not sure how relevant a paper that old (relatively, I hasten to add) is.
What are you talking about? Fugaku is the opposite of that, each CPU chip is 48 cores with 512 bit wide SVE(arm version of AVX512). They deliberately went for something easier to program, that didn't require doing the CPU/GPU dance.
ARM/RiscV with many cores...that's what i wrote, if you don't need a gpu you don't need one, if you need SVE you integrate it or use a co-Processor :)
SVE isn't in a co-processor, I guess is the point. There's a lot more to Fugaku than SVE (whether or not you think that's a version of avx512), though. No DDR is suggestive.