Looks like a great use case for AI. Set up the logical specification and constraints and let the AI find the optimal sequence of SIMD operations to fulfill the requirements.
No, there are decades of compiler literature for solving this problem.
Understanding SIMD: Infinite complexity of trivial problems
31–40 of 127 posts
Re: Understanding SIMD: Infinite complexity of trivial problems
#32C# vectors do a great job of simplifying those intrinsics in a safe and portable manner.
There are dozens of libraries, frameworks, and compiler toolchains that try to abstract away SIMD capabilities, but I don't think it's a great approach. The only 2 approaches that still make sense to me: A. Writing serial vectorization-aware code in a native compiled language, hoping your compiler will auto-vectorize. B. Implementing natively for every hardware platform, as the ISA differences are too big to efficien…
Numerics in .NET are not a high-level abstraction and do out of box what many mature vectorized libraries end up doing themselves - there is significant overlap between NEON, SSE* and, if we overlook vector width, AVX2/512 and WASMs PackedSIMD.
.NET has roughly three vector APIs:
- Vector which is platform-defined width vector that exposes common set of operations
- Vector64/128/256/512 which has wider API than the previous one
- Platform intrinsics - basically immintrin.h
Notably, platform intrinsics use respective VectorXXX types which allows to write common parts of the algorithm in a portable way and apply platform intrinsics in specific areas where it makes sense. Also some method have 'Unsafe' and 'Native' variants to allow for vector to exhibit platform-specific behavior like shuffles since in many situations this is still the desired output for the common case.
The .NET's compiler produces competitive with GCC and sometimes Clang codegen for these. It's gotten particularly good at lowering AVX512.
Re: Understanding SIMD: Infinite complexity of trivial problems
#33I see a lot of "just use the GPU" and you'd often be right. SIMD on the CPU is most compelling to me due to the latency characteristics. You are nanoseconds away from the control flow. If the GPU needs some updated state regarding the outside world, it takes significantly longer to propagate this information. For most use cases, the GPU will win the trade off. But, there is a reason you don't hear much about systems…
Do Apple's chips (M1 etc) change this at all, since they share memory with the GPU?
Re: Understanding SIMD: Infinite complexity of trivial problems
#34Earlier quoted context omitted.
Maximum performance? Okay, you'll have to upgrade to ballot instructions or whatever and rearchitect your algorithms. (Or other wavefront / voting / etc. etc. new instructions that have been invented. Especially those 4x4 matrix multiplication AI instructions). But CUDA -> PTX intermediate code has allowed for significantly more flexibility. For crying out loud, the entire machine code (aka SASS) of NVidia GPUs has b…
Intel code from 15 years ago also runs today. But it will not use AVX512. Which is the same with PTX, right? If you didn't use the tensor core instructions or wavefront voting in the CUDA code, the PTX generated from it will not either, and NVIDIA will not magically add those capabilities in when compiling to SASS. Maybe it remains competitive because the code is inherently parallel anyway, so it will naturally scale…
It's not the same. AVX2 instructions haven't changed and never will change.
In contrast, NVidia can go from 64-bit instruction bundles to 128-bit machine code (96-bit instruction + 32-bit control information) between Pascal (aka PTX Compute Capacity 5) and Voltage (aka PTX Compute Capacity 7) and all the old PTX code just autocompiles to the new assembly instruction format and takes advantage of all the new memory barriers added in Volta.
Having a PTX translation later is a MAJOR advantage for the NVidia workflow.
Re: Understanding SIMD: Infinite complexity of trivial problems
#35I see a lot of "just use the GPU" and you'd often be right. SIMD on the CPU is most compelling to me due to the latency characteristics. You are nanoseconds away from the control flow. If the GPU needs some updated state regarding the outside world, it takes significantly longer to propagate this information. For most use cases, the GPU will win the trade off. But, there is a reason you don't hear much about systems…
Despite my 'Use a GPU' post below, you are absolutely correct.
Maximizing performance on a CPU today requires all the steps in the above article, and the article is actually very well written with regards to the 'mindset' needed to tackle a problem such as this.
It's a great article for people aiming to maximize the performance on Intel or AMD systems.
------
CPUs have the memory capacity advantage and will continue to hold said advantage for the foreseeable future (despite NVidias NVLink and other techs to try to bridge the gap).
And CPU code remains far easier than learning CUDA, despite how hard these AVX intrinsics are in comparison to CUDA.
Re: Understanding SIMD: Infinite complexity of trivial problems
#36Looks like a great use case for AI. Set up the logical specification and constraints and let the AI find the optimal sequence of SIMD operations to fulfill the requirements.
No, there are decades of compiler literature for solving this problem.
That's even better then. Just let the AI read the literature and write the optimal compiler.
Re: Understanding SIMD: Infinite complexity of trivial problems
#37Earlier quoted context omitted.
No, there are decades of compiler literature for solving this problem.
That's even better then. Just let the AI read the literature and write the optimal compiler.
It would probably be easier to clone the existing repository than get an llm to regurgitate llvm.
Re: Understanding SIMD: Infinite complexity of trivial problems
#38Re: Understanding SIMD: Infinite complexity of trivial problems
#39Intel needs to see what has happened to their AVX instructions and why NVidia has taken over. If you just wrote your SIMD in CUDA 15 years ago, NVidia compilers would have given you maximum performance across all NVidia GPUs rather than being forced to write and rewrite in SSE vs AVX vs AVX512. GPU SIMD is still SIMD. Just... better at it. I think AMD and Intel GPUs can keep up btw. But software advantage and long te…
How much of this is because CUDA is designed for GPU execution and because the GPU ISA isn’t a stable interface? E.g. new GPU instructions can be utilized by new CUDA compilers for new hardware because the code wasn’t written to a specific ISA? Also, don’t people fine tune GPU kernels per architecture manually (either by hand or via automated optimizers that test combinations in the configuration space)?
NVidia PTX is a very stable interface.
And the PTX to SASS compiler DOES a degree of automatic fine tuning between architectures. Nothing amazing or anything, but it's a minor speed boost that has made PTX just a easier 'assembly-like language' to build on top of.
Re: Understanding SIMD: Infinite complexity of trivial problems
#40You can simplify the 2x sqrts as sqrt(a*b), overall less operations so perhaps more accurate. It would also let you get rid of the funky lane swivels.
As this would only use 1 lane, perhaps if you have multiple of these to normalize, you could vectorize it.