Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding SIMD: Infinite complexity of trivial problems

modular.com

Post reply on HN
Print view View original

61–70 of 127 posts

Re: Understanding SIMD: Infinite complexity of trivial problems

#61

Post by snihalani »

dragontamer wrote: post #6

Intel needs to see what has happened to their AVX instructions and why NVidia has taken over. If you just wrote your SIMD in CUDA 15 years ago, NVidia compilers would have given you maximum performance across all NVidia GPUs rather than being forced to write and rewrite in SSE vs AVX vs AVX512. GPU SIMD is still SIMD. Just... better at it. I think AMD and Intel GPUs can keep up btw. But software advantage and long te…

> software methodology to interact with SIMD using GPU-like languages should be a priority.

What's your opinion on sycl?

https://www.khronos.org/sycl/

Re: Understanding SIMD: Infinite complexity of trivial problems

#62

Post by bjourne »

dragontamer wrote: post #9
ashvardanian wrote: post #8

Earlier quoted context omitted.

That may have been my mistake. I use super & hyper interchangeably and don't always notice :) PS: Should be an easy patch, will update!

Maybe not. Superscalar is when say... Think of the following assembly code. Add r1, r2 Sub r3, r4 And the add and subtract both happen on the same clock tick. The important thing is that a modern CPU core (and even GPU core) have multiple parallel ALU pipelines inside of them. Because r1, r2, r3 and r4 are fully independent, a modern CPU can detect the potential parallelism here and act in parallel. After CPUs master…

Agree with this. Calling SIMD superscalar is a misnomer since it is single instruction (multiple data) with very wide data paths. Superscalar implies multiple different instructions in parallel, such as adding a pair of numbers, while subtracting another pair (or even dividing).

Re: Understanding SIMD: Infinite complexity of trivial problems

#63

Post by ip26 »

dragontamer wrote: post #6

Intel needs to see what has happened to their AVX instructions and why NVidia has taken over. If you just wrote your SIMD in CUDA 15 years ago, NVidia compilers would have given you maximum performance across all NVidia GPUs rather than being forced to write and rewrite in SSE vs AVX vs AVX512. GPU SIMD is still SIMD. Just... better at it. I think AMD and Intel GPUs can keep up btw. But software advantage and long te…

Is CUDA not more analogous to using MKL, rather than AVX?

Re: Understanding SIMD: Infinite complexity of trivial problems

#65

Post by [deleted] »

bob1029 wrote: post #23

I see a lot of "just use the GPU" and you'd often be right. SIMD on the CPU is most compelling to me due to the latency characteristics. You are nanoseconds away from the control flow. If the GPU needs some updated state regarding the outside world, it takes significantly longer to propagate this information. For most use cases, the GPU will win the trade off. But, there is a reason you don't hear much about systems…

[deleted]

Re: Understanding SIMD: Infinite complexity of trivial problems

#67

Post by harry8 »

ashvardanian wrote: post #54
a_gopher wrote: post #51

Earlier quoted context omitted.

my thoughts exactly - crazy to know all these arcane SIMD opcodes but not know basic maths!!

Square root computation can be tricky, often relying on approximations. These approximations tend to perform best for mid-range values, while accuracy can degrade for very large or very small values. With this in mind, a product of roots is generally more accurate than a root of products. From a SIMD perspective, it’s worth noting that on most platforms, the cost of computing one square root or two is the same. On mo…

> it’s usually more productive to ask for clarification than to assume ignorance ;)

Good reminder for me and anyone else right there, nicely put.

Re: Understanding SIMD: Infinite complexity of trivial problems

#68

Post by creato »

dragontamer wrote: post #17
pyrolistical wrote: post #14

Earlier quoted context omitted.

I wish hardware exposed an api that allowed us to submit a tree of instructions so the hardware doesn’t need figure out which instructions are independent. Lots of this kind of work can be done during compilation but cannot be communicated to hardware due to code being linear

That's called VLIW and Intel Itanium is considered one of the biggest chip failures of all time. There is an argument that today's compilers are finally good enough for VLIW to go mainstream, but good luck convincing anyone in today's market to go for it. ------ A big problem with VLIW is that it's impossible to predict L1, L2, L3 or DRAM access. Meaning all loads/stores are impossible to schedule by the compiler. NV…

VLIW is still in use in multiple DSP products on the market today, and they are good successful products in their niche.

They work very well if your code can be written as a loop without branches (or very limited branches) in the body, and a lot of instruction level parallelism in the body.

Unfortunately for Intel, most code doesn't look like that. But for most workloads that happen to also be a good case for SIMD, it is (can be) great.

Re: Understanding SIMD: Infinite complexity of trivial problems

#69

Post by jabl »

dragontamer wrote: post #6

Intel needs to see what has happened to their AVX instructions and why NVidia has taken over. If you just wrote your SIMD in CUDA 15 years ago, NVidia compilers would have given you maximum performance across all NVidia GPUs rather than being forced to write and rewrite in SSE vs AVX vs AVX512. GPU SIMD is still SIMD. Just... better at it. I think AMD and Intel GPUs can keep up btw. But software advantage and long te…

Sometimes I wonder about an alternative history scenario where CPU ISA's would have chosen a SIMT style model instead of SIMD. "Just" have something like fork/join instructions to start/stop vector mode, otherwise use the standard scalar instructions in both scalar and vector mode. Would have avoided a lot of combinatorial explosion in instructions. (of course you'd have to do something for cross-lane operations, and later tensor instructions etc.)

Re: Understanding SIMD: Infinite complexity of trivial problems

#70

Post by Earw0rm »

pjmlp wrote: post #15
dragontamer wrote: post #6

Intel needs to see what has happened to their AVX instructions and why NVidia has taken over. If you just wrote your SIMD in CUDA 15 years ago, NVidia compilers would have given you maximum performance across all NVidia GPUs rather than being forced to write and rewrite in SSE vs AVX vs AVX512. GPU SIMD is still SIMD. Just... better at it. I think AMD and Intel GPUs can keep up btw. But software advantage and long te…

It is worse than that, given that AVX is the survivor from Larrabee great plan to kill GPUs. Larrabee was going to take over it all, as I enjoyed its presentation at GDCE 2009.

And a few years later, Intel said we'd get AVX512 on everything by 2016, and that the instruction encoding supported a future extension to 1024.

And then the Skylake and Cannon Lake debacle..

First they pulled it from the consumer chips a fairly short time before launch. Then the server chips it was present in would downclock aggressively when you did use it, so you could get at best maybe 40% more performance, certainly far from the 2x+ it promised.

Ten years on and the AMD 9950X does a pretty good job with it, however.

Post reply on HN