Linus Torvalds on AVX512 • Hacker News Board

Re: Linus Torvalds on AVX512

#11

Post by gridlockd » Sun, Jul 12, 2020, 8:55 AM UTC

"I've said this before, and I'll say it again: in the heyday of x86, when Intel was laughing all the way to the bank and killing all their competition, absolutely everybody else did better than Intel on FP loads. Intel's FP performance sucked (relatively speaking), and it matter not one iota.

Because absolutely nobody cares outside of benchmarks."

That was back in the stone age when a lot of applications for FP math weren't mainstream. Most of AVX-512 doesn't even concern FP, there's lots of integer and bit twiddling stuff there.

Furthermore, people really do care about these benchmarks. It influences their purchasing, which is really the thing that matters most to Intel. A lot of people don't actually care about hypothetical security issues or the fact that the CPU is 14nm when it still outperforms 7nm in single-threaded code.

Also, it's not like you can just trade off IPC or extra cores for wider SIMD. It's not like "just add more cores" is just as good for throughput, otherwise GPUs wouldn't exist. Wider SIMD is cheap in terms of die area, for the throughput it gives you.

Lastly, these are just instructions, nothing says that an AVX-512 instruction needs to go through a physical 512-bit wide unit, it just says that you can take advantage of those semantics, if possible.

Re: Linus Torvalds on AVX512

#12

Post by abainbridge » Sun, Jul 12, 2020, 8:59 AM UTC

What are the forces in chip design that are at play here? Over the last 10-15 years, fabs have continued to fit more and more logic gates per unit area, but haven't reduced the power consumption per gate as much. As a result, if you fill your modern chip with compute gates, you cannot use them all at once because the chip will melt. Or at least you can't have them all running at max clock rates. One solution is to increase the proportion of the chip used for SRAM (it uses less power per unit area than compute gates), this is what Graphcore have done. Another is to put down multiple different compute blocks, each designed for a different purpose, and only use them a-few-at-a-time. The big-little Arm designs in smartphones are an example of that. But I feel like AVX512 might be an example too. When they add ML accelerator blocks next, they also will not be able to be used flat out at the same time as the rest of the cores' resources.

I'm sure Intel should fix the problems Linus is complaining about, but I feel like chip vendors are being forced into this "add special purpose blocks" approach, as the only way to make their new chips better than their old ones.

Re: Linus Torvalds on AVX512

#13

Post by superjan » Sun, Jul 12, 2020, 9:01 AM UTC

throwaway_pdp09 wrote: post #9

The very wide AVX stuff with integer ops, like these from wiki: - AVX-512 Byte and Word Instructions (BW) – extends AVX-512 to cover 8-bit and 16-bit integer operations[3] - AVX-512 Integer Fused Multiply Add (IFMA) - fused multiply add of integers using 52-bit precision. could be very useful. I could have done with those recently. They also don't (AFAIK) cause cpu scaling (polite term for downclocking). He may well…

52 bit precision? typo?

Re: Linus Torvalds on AVX512

#14

Post by pjmlp » Sun, Jul 12, 2020, 9:03 AM UTC

RantyDave wrote: post #2

Are Intel just delaying the inevitable? Is it safe to say (even today) that a slow GPU will crunch big matrices faster than a fast CPU? And that's before we get to price/performance. So all that's left is the bottleneck around PCIe which, in theory, leaves the CPU with an advantage only for small datasets - which we don't really care about anyway (because they happen quickly). Maybe the tradeoff is somewhere interest…

AVX is the surviving heritage from Larrabee, the CPU that would outtake GPGPU and so far has failed to do so.

The only thing that Intel has going for their GPUs is that as typically happens with the underdog companies, they decided to play nice with FOSS drivers and with integrated GPUs they own the low budget laptop market.

Everyone that has done any serious 3D programming is painfully aware how bad their OpenGL drivers used to be, they even used to fake OpenGL queries confirming features as supported, when they were actually implemented in software, thus making some games unusable.

That is why they started the campaign about optimizing games for Intel GPUs, and how to make best use of Graphics Profile Analyser, which ironically in the old days was DirectX only.

The bottleneck you mention is only an issue when there isn't any shared memory available, if the hardware allows for unified memory models then there is no data transfer and the GPU can work right way, naturally there are some synchronization points that need to happen still.

Re: Linus Torvalds on AVX512

#15

Post by gridlockd » Sun, Jul 12, 2020, 9:04 AM UTC

throwaway_pdp09 wrote: post #9

The very wide AVX stuff with integer ops, like these from wiki: - AVX-512 Byte and Word Instructions (BW) – extends AVX-512 to cover 8-bit and 16-bit integer operations[3] - AVX-512 Integer Fused Multiply Add (IFMA) - fused multiply add of integers using 52-bit precision. could be very useful. I could have done with those recently. They also don't (AFAIK) cause cpu scaling (polite term for downclocking). He may well…

If he was right with FP, he'd know better than the business analysts at Intel. Instead, his opinion is based on what the market looked like thirty years ago.

Nine years ago, AMD tested the hypothesis that really more "cores" and higher integer throughput were all that was needed and that FP performance didn't matter. The resulting architecture (Bulldozer) was a near-fatal disaster. It didn't even work out in the datacenter, where you might expect that hypothesis to hold.

Re: Linus Torvalds on AVX512

#16

Post by m0xte » Sun, Jul 12, 2020, 9:04 AM UTC

nix23 wrote: post #8

RantyDave wrote: post #2

Are Intel just delaying the inevitable? Is it safe to say (even today) that a slow GPU will crunch big matrices faster than a fast CPU? And that's before we get to price/performance. So all that's left is the bottleneck around PCIe which, in theory, leaves the CPU with an advantage only for small datasets - which we don't really care about anyway (because they happen quickly). Maybe the tradeoff is somewhere interest…

Yes the future is probably ARM/RiscV with many cores + GPU + some AI/ML/FPGA/Whatever co-Processor.

So what Apple are doing then.

Re: Linus Torvalds on AVX512

#17

Post by protomyth » Sun, Jul 12, 2020, 9:05 AM UTC

TazeTSchnitzel wrote: post #5

rbanffy wrote: post #4

I for one would be delighted by having more caches or wider backends instead of AVX512, but I don't want SIMD to be pushed into GPUs. It'd be better to do the reverse - to push forward the asymmetric core idea and move more GPU functionality into lots of simpler cores tuned for SIMD at the cost of single thread performance.

So, Larrabee?

Cell with a saner bus/memory access?

Re: Linus Torvalds on AVX512

#18

Post by pjmlp » Sun, Jul 12, 2020, 9:07 AM UTC

abainbridge wrote: post #12

What are the forces in chip design that are at play here? Over the last 10-15 years, fabs have continued to fit more and more logic gates per unit area, but haven't reduced the power consumption per gate as much. As a result, if you fill your modern chip with compute gates, you cannot use them all at once because the chip will melt. Or at least you can't have them all running at max clock rates. One solution is to in…

The main problem is software, with GPGPUs you need to explicitly program for them, while with stuff like AVX there is this implicit hope that you just code as always and the compiler will take care of the rest via auto-vectorization and PhD level optimization algorithms.

Because outside artificial intelligence, graphics and audio, there is little else that common applications would use the GPGPU for, so the large majority of software developers keeps ignoring heterogeneous programming models.

Re: Linus Torvalds on AVX512

#19

Post by gridlockd » Sun, Jul 12, 2020, 9:11 AM UTC

abainbridge wrote: post #12

What are the forces in chip design that are at play here? Over the last 10-15 years, fabs have continued to fit more and more logic gates per unit area, but haven't reduced the power consumption per gate as much. As a result, if you fill your modern chip with compute gates, you cannot use them all at once because the chip will melt. Or at least you can't have them all running at max clock rates. One solution is to in…

AVX is not a "special purpose block", it's Intel's answer to not adding special purpose blocks on customer demand, like you can do with ARM.

Crypto or video decoding comes to mind, those would be much faster with dedicated silicon, but more general AVX instructions can get you halfway there. Well, maybe a quarter. People point out that AVX uses a lot of power, but they ignore that the same algorithm running instead on more but simpler cores would use even more power.

Re: Linus Torvalds on AVX512

#20

Post by raverbashing » Sun, Jul 12, 2020, 9:14 AM UTC

FP matters (especially with SIMD)

It matters to image/video/audio processing

It matters to simulations

It matters to 3D models/rendering

It matters to games

So it's not "just benchmarks", people actually want to do stuff with it

Sure, AVX512 might not be the greatest way of doing it, and it might be better to just make the existing instructions go faster, that might work