51–60 of 127 posts
You can simplify the 2x sqrts as sqrt(a*b), overall less operations so perhaps more accurate. It would also let you get rid of the funky lane swivels. As this would only use 1 lane, perhaps if you have multiple of these to normalize, you could vectorize it.
Earlier quoted context omitted.
All we have to do is ascribe magical properties to AI and we can solve anything as if P=NP!
Those distinction are irrelevant for an AI because it is a pure form of intelligence that simply computes answers without worrying about P or NP complexity classes.
B-
Intel needs to see what has happened to their AVX instructions and why NVidia has taken over. If you just wrote your SIMD in CUDA 15 years ago, NVidia compilers would have given you maximum performance across all NVidia GPUs rather than being forced to write and rewrite in SSE vs AVX vs AVX512. GPU SIMD is still SIMD. Just... better at it. I think AMD and Intel GPUs can keep up btw. But software advantage and long te…
> Intel ISPC is a great project btw if you need high level code that targets SSE, AVX, AVX512 and even ARM NEON It's pretty funny how NEON ended up in there. A former Intel employee decided to implement it for fun and submitted it as a pull request, which Intel quietly ignored for obvious reasons, but then another former Intel employee who still had commit rights merged the PR, and the optics of publicly reverting it…
You can simplify the 2x sqrts as sqrt(a*b), overall less operations so perhaps more accurate. It would also let you get rid of the funky lane swivels. As this would only use 1 lane, perhaps if you have multiple of these to normalize, you could vectorize it.
my thoughts exactly - crazy to know all these arcane SIMD opcodes but not know basic maths!!
From a SIMD perspective, it’s worth noting that on most platforms, the cost of computing one square root or two is the same. On modern x86 server CPUs, for instance, you can calculate up to 8 double-precision roots in parallel with identical latency. So there’s no additional cost in terms of performance.
I hope this sheds some light on the design of my code.
PS: In a previous life, I did research in Astro- and Plasma Physics. While I don’t claim to remember all the Math, it’s usually more productive to ask for clarification than to assume ignorance ;)
Earlier quoted context omitted.
Do Apple's chips (M1 etc) change this at all, since they share memory with the GPU?
Apple chips share the same physical memory between the GPU and the CPU. Still, they don't have USM/UVM (Unified Shared Memory/Unified Virtual Memory), that is, the GPU and the CPU can't access the same data concurrently and easily. Programs must map/unmap pages to control which device accesses it, and that's a very expensive operation.
Earlier quoted context omitted.
Intel code from 15 years ago also runs today. But it will not use AVX512. Which is the same with PTX, right? If you didn't use the tensor core instructions or wavefront voting in the CUDA code, the PTX generated from it will not either, and NVIDIA will not magically add those capabilities in when compiling to SASS. Maybe it remains competitive because the code is inherently parallel anyway, so it will naturally scale…
It's not the same. AVX2 instructions haven't changed and never will change. In contrast, NVidia can go from 64-bit instruction bundles to 128-bit machine code (96-bit instruction + 32-bit control information) between Pascal (aka PTX Compute Capacity 5) and Voltage (aka PTX Compute Capacity 7) and all the old PTX code just autocompiles to the new assembly instruction format and takes advantage of all the new memory ba…
Earlier quoted context omitted.
Intel code from 15 years ago also runs today. But it will not use AVX512. Which is the same with PTX, right? If you didn't use the tensor core instructions or wavefront voting in the CUDA code, the PTX generated from it will not either, and NVIDIA will not magically add those capabilities in when compiling to SASS. Maybe it remains competitive because the code is inherently parallel anyway, so it will naturally scale…
It's not the same. AVX2 instructions haven't changed and never will change. In contrast, NVidia can go from 64-bit instruction bundles to 128-bit machine code (96-bit instruction + 32-bit control information) between Pascal (aka PTX Compute Capacity 5) and Voltage (aka PTX Compute Capacity 7) and all the old PTX code just autocompiles to the new assembly instruction format and takes advantage of all the new memory ba…
[flagged]
Earlier quoted context omitted.
It is worse than that, given that AVX is the survivor from Larrabee great plan to kill GPUs. Larrabee was going to take over it all, as I enjoyed its presentation at GDCE 2009.
I mean, 288-E Core Xeons are about to ship. Xeon 6900 series, right? (Estimated to ship in Q1 2025) So Larrabee lives on for... some reason. These E cores are well known to be modified Intel Atom cores and those were modified Xeon Phi cores which were Larrabee based. Just with.... AVX512 being disabled. (Lost when Xeon Phi turned into Intel Atoms). Intels technical strategy is completely bonkers. In a bad way. Intel…
Ironically, AMD waited so long to implement AVX-512, but now has it on both server and mobile chips (natively and 256 bit emulation, respectively). Intel started the whole thing, has a very fragmented stack and is now preparing those E cores with even more new extensions.
Most importantly for Search and AI, it adds AVX_VNNI, which can be used for faster 8-bit integer dot-products: https://github.com/ashvardanian/SimSIMD/blob/75c426fb190a9d4...
Would be interesting to see how matrix multiplication throughput will differ between AVX-512-capable P cores and a larger quantity of AVX_VNNI-capable E cores!
> SIMD instructions are complex, and even Arm is starting to look more “CISCy” than x86! Thank you for saying it out loud. XLAT/XLATB of x86 is positively tame compared to e.g. vrgatherei16.vv/vrgather.vv.