Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding SIMD: Infinite complexity of trivial problems

modular.com

Post reply on HN
Print view View original

11–20 of 127 posts

Re: Understanding SIMD: Infinite complexity of trivial problems

#11

Post by jsheard »

dragontamer wrote: post #6

Intel needs to see what has happened to their AVX instructions and why NVidia has taken over. If you just wrote your SIMD in CUDA 15 years ago, NVidia compilers would have given you maximum performance across all NVidia GPUs rather than being forced to write and rewrite in SSE vs AVX vs AVX512. GPU SIMD is still SIMD. Just... better at it. I think AMD and Intel GPUs can keep up btw. But software advantage and long te…

> Intel ISPC is a great project btw if you need high level code that targets SSE, AVX, AVX512 and even ARM NEON

It's pretty funny how NEON ended up in there. A former Intel employee decided to implement it for fun and submitted it as a pull request, which Intel quietly ignored for obvious reasons, but then another former Intel employee who still had commit rights merged the PR, and the optics of publicly reverting it would be even worse than stonewalling so Intel begrudgingly let it stand (but they did revoke that devs commit rights).

https://pharr.org/matt/blog/2018/04/29/ispc-retrospective

Re: Understanding SIMD: Infinite complexity of trivial problems

#13

Post by juancn »

The main problem is that there are no good abstractions in popular programming languages to take advantage of SIMD extensions.

Also, the feature set being all over the place (e.g. integer support is fairly recent) doesn't help either.

ISPC is a good idea, but execution is meh... it's hard to setup and integrate.

Ideally you would want to be able to easily use this from other popular languages, like Java, Python, Javascript, without having to resort to linking a library written in C/C++.

Granted, language extensions may be required to approach something like that in an ergonomic way, but most somehow end up just mimicking what C++ does and expose a pseudo assembler.

Re: Understanding SIMD: Infinite complexity of trivial problems

#14

Post by pyrolistical »

dragontamer wrote: post #9
ashvardanian wrote: post #8

Earlier quoted context omitted.

That may have been my mistake. I use super & hyper interchangeably and don't always notice :) PS: Should be an easy patch, will update!

Maybe not. Superscalar is when say... Think of the following assembly code. Add r1, r2 Sub r3, r4 And the add and subtract both happen on the same clock tick. The important thing is that a modern CPU core (and even GPU core) have multiple parallel ALU pipelines inside of them. Because r1, r2, r3 and r4 are fully independent, a modern CPU can detect the potential parallelism here and act in parallel. After CPUs master…

I wish hardware exposed an api that allowed us to submit a tree of instructions so the hardware doesn’t need figure out which instructions are independent.

Lots of this kind of work can be done during compilation but cannot be communicated to hardware due to code being linear

Re: Understanding SIMD: Infinite complexity of trivial problems

#15

Post by pjmlp »

dragontamer wrote: post #6

Intel needs to see what has happened to their AVX instructions and why NVidia has taken over. If you just wrote your SIMD in CUDA 15 years ago, NVidia compilers would have given you maximum performance across all NVidia GPUs rather than being forced to write and rewrite in SSE vs AVX vs AVX512. GPU SIMD is still SIMD. Just... better at it. I think AMD and Intel GPUs can keep up btw. But software advantage and long te…

It is worse than that, given that AVX is the survivor from Larrabee great plan to kill GPUs.

Larrabee was going to take over it all, as I enjoyed its presentation at GDCE 2009.

Re: Understanding SIMD: Infinite complexity of trivial problems

#16

Post by pjmlp »

juancn wrote: post #13

The main problem is that there are no good abstractions in popular programming languages to take advantage of SIMD extensions. Also, the feature set being all over the place (e.g. integer support is fairly recent) doesn't help either. ISPC is a good idea, but execution is meh... it's hard to setup and integrate. Ideally you would want to be able to easily use this from other popular languages, like Java, Python, Java…

The best is the GPU programming approach, with specialised languages

Just like using SQL is much more sane than low level C APIs to handle BTree nodes.

The language extensions help, but code still requires too much low level expertise, with algorithms and data structures having to take SIMD/MIMD capabilities into account anyway.

Re: Understanding SIMD: Infinite complexity of trivial problems

#17

Post by dragontamer »

pyrolistical wrote: post #14
dragontamer wrote: post #9

Earlier quoted context omitted.

Maybe not. Superscalar is when say... Think of the following assembly code. Add r1, r2 Sub r3, r4 And the add and subtract both happen on the same clock tick. The important thing is that a modern CPU core (and even GPU core) have multiple parallel ALU pipelines inside of them. Because r1, r2, r3 and r4 are fully independent, a modern CPU can detect the potential parallelism here and act in parallel. After CPUs master…

I wish hardware exposed an api that allowed us to submit a tree of instructions so the hardware doesn’t need figure out which instructions are independent. Lots of this kind of work can be done during compilation but cannot be communicated to hardware due to code being linear

That's called VLIW and Intel Itanium is considered one of the biggest chip failures of all time.

There is an argument that today's compilers are finally good enough for VLIW to go mainstream, but good luck convincing anyone in today's market to go for it.

------

A big problem with VLIW is that it's impossible to predict L1, L2, L3 or DRAM access. Meaning all loads/stores are impossible to schedule by the compiler.

NVidia has interesting barriers that get compiled into its SASS (a level lower than PTX assembly). These barriers seem to allow the compiler to assist in the dependency management process but ultimately still require a decoder in the NVidia core final level before execution.

Re: Understanding SIMD: Infinite complexity of trivial problems

#18

Post by [deleted] »

pyrolistical wrote: post #14
dragontamer wrote: post #9

Earlier quoted context omitted.

Maybe not. Superscalar is when say... Think of the following assembly code. Add r1, r2 Sub r3, r4 And the add and subtract both happen on the same clock tick. The important thing is that a modern CPU core (and even GPU core) have multiple parallel ALU pipelines inside of them. Because r1, r2, r3 and r4 are fully independent, a modern CPU can detect the potential parallelism here and act in parallel. After CPUs master…

I wish hardware exposed an api that allowed us to submit a tree of instructions so the hardware doesn’t need figure out which instructions are independent. Lots of this kind of work can be done during compilation but cannot be communicated to hardware due to code being linear

[deleted]

Re: Understanding SIMD: Infinite complexity of trivial problems

#19

Post by dragontamer »

dist-epoch wrote: post #10
dragontamer wrote: post #6

Intel needs to see what has happened to their AVX instructions and why NVidia has taken over. If you just wrote your SIMD in CUDA 15 years ago, NVidia compilers would have given you maximum performance across all NVidia GPUs rather than being forced to write and rewrite in SSE vs AVX vs AVX512. GPU SIMD is still SIMD. Just... better at it. I think AMD and Intel GPUs can keep up btw. But software advantage and long te…

> If you just wrote your SIMD in CUDA 15 years ago, NVidia compilers would have given you maximum performance across all NVidia GPUs That's not true. For maximum performance you need to tweak the code to a particular GPU model/architecture. Intel has SSE/AVX/AVX2/AVX512, but CUDA has like 10 iterations of this (increasing capabilities). Code written 15 years ago would not use modern capabilities, like more flexible m…

Maximum performance? Okay, you'll have to upgrade to ballot instructions or whatever and rearchitect your algorithms. (Or other wavefront / voting / etc. etc. new instructions that have been invented. Especially those 4x4 matrix multiplication AI instructions).

But CUDA -> PTX intermediate code has allowed for significantly more flexibility. For crying out loud, the entire machine code (aka SASS) of NVidia GPUs has been cycled out at least 4 times in the past decade (128-bit bundles, changes to instruction formats, acquire/release semantics, etc etc)

It's amazing what backwards compatibility NVidia has achieved in the past 15 years thanks to this architecture. SASS changes so dramatically from generation to generation but the PTX intermediate code has stayed highly competitive.

Re: Understanding SIMD: Infinite complexity of trivial problems

#20

Post by dragontamer »

pjmlp wrote: post #15
dragontamer wrote: post #6

Intel needs to see what has happened to their AVX instructions and why NVidia has taken over. If you just wrote your SIMD in CUDA 15 years ago, NVidia compilers would have given you maximum performance across all NVidia GPUs rather than being forced to write and rewrite in SSE vs AVX vs AVX512. GPU SIMD is still SIMD. Just... better at it. I think AMD and Intel GPUs can keep up btw. But software advantage and long te…

It is worse than that, given that AVX is the survivor from Larrabee great plan to kill GPUs. Larrabee was going to take over it all, as I enjoyed its presentation at GDCE 2009.

I mean, 288-E Core Xeons are about to ship. Xeon 6900 series, right? (Estimated to ship in Q1 2025)

So Larrabee lives on for... some reason. These E cores are well known to be modified Intel Atom cores and those were modified Xeon Phi cores which were Larrabee based.

Just with.... AVX512 being disabled. (Lost when Xeon Phi turned into Intel Atoms).

Intels technical strategy is completely bonkers. In a bad way. Intel invented all this tech 10 to 20 years ago but fails to have a cohesive strategy to bring it to market. There's clearly smart people there but somehow all the top level decisions are just awful

Post reply on HN