Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SSE: mind the gap

fgiesen.wordpress.com

Post reply on HN
Print view View original

31–35 of 35 posts

Re: SSE: mind the gap

#31

Post by cm3 »

vardump wrote: post #30
cm3 wrote: post #28

Earlier quoted context omitted.

So all this speculative business is more about using idle bits of the chip to warm it up for either branch to be taken, thereby reducing some of the time for that code to execute, but any and all op that would actually modify memory or access stuff on the bus is exempt from speculation. About right?

> About right? Close. There are two kinds of speculative execution. 1) CPU guesses (branch predicts) one path by default. If it guesses wrong, it'll need to throw away speculative results and to execute the alternative path. No speculative state is leaked to other CPU cores or memory (writes or I/O). Typically CPUs guess right way over 99% of time -- there are of course cases when prediction fails, sometimes patholog…

How does it determine it was wrong without checking the condition? Or does it?

Re: SSE: mind the gap

#32

Post by vardump »

cm3 wrote: post #31
vardump wrote: post #30

Earlier quoted context omitted.

> About right? Close. There are two kinds of speculative execution. 1) CPU guesses (branch predicts) one path by default. If it guesses wrong, it'll need to throw away speculative results and to execute the alternative path. No speculative state is leaked to other CPU cores or memory (writes or I/O). Typically CPUs guess right way over 99% of time -- there are of course cases when prediction fails, sometimes patholog…

How does it determine it was wrong without checking the condition? Or does it?

It catches up with the branch condition with some delay, because CPUs have a lot of pipelined stages. CPU is not executing one or two instructions at a time, but a window of 10-50 (guesstimate, maybe more) instructions over roughly 15 clock cycles -- pipeline depth.

So after it knows the condition result that determines the taken branch say 15 cycles later, it compares the guess to the actual path that needs to be taken. If they agree, speculative results are marked valid. If not, they're thrown away, CPU pipeline is flushed and execution starts again from the other path.

CPUs also have hundreds (current crop is about 200 uops (read: instructions) reorder buffer (ROB), Intel Haswell has 192), where CPU tries to sort the cross instruction dependencies in an order that's faster to execute. Deep pipeline means if it can't reorder the instructions, it'll have to stall waiting for the earlier results -- it just doesn't know the value of certain register (or cached memory location), until the earlier computation dependency chain is finished.

They're complicated and weird machines. They don't really execute the code sequentially at all, just make it look like as if they did.

Everything I said above is oversimplified. I left out register renaming, cross core / CPU socket cache coherency -- and so much more. I can't really say I completely understand the beast myself.

Re: SSE: mind the gap

#33

Post by wmu »

exDM69 wrote: post #16
wmu wrote: post #4

Earlier quoted context omitted.

Not sure about "single code path". Differences amid SIMD flavors are significant, there are cases when translation one-to-one is either impossible or unpractical. A bright example are AVX2 instructions operating on 128-bit lanes rather whole 256-bit registers. And wrappers exists in the C++ ecosystem, C programmers are stuck to intrinsics.

> And wrappers exists in the C++ ecosystem, C programmers are stuck to intrinsics. If you can accept working with GNU extensions that are available in recent-ish GCC and Clang (but not MSVC, not sure about Intel ICC), there are pretty nice vector extensions [0]. With them you can get standard binary operators working for arithmetic (+,-,*,/ etc) and shuffling with __builtin_shuffle. These are CPU independent, the sam…

> If you can accept working with GNU extensions that are available in recent-ish GCC and Clang

I do my private project in C++ so it's not a case, but at my current company we use also MSVC. I wish we could abandon that compiler and work with GCC or clang only.

> However, for a lot of SIMD tasks I encounter, just basic arithmetic + shuffles is more than 80% of what I need.

Your remaining 20% is my 80%. :)

Re: SSE: mind the gap

#34

Post by exDM69 »

wmu wrote: post #33
exDM69 wrote: post #16

Earlier quoted context omitted.

> And wrappers exists in the C++ ecosystem, C programmers are stuck to intrinsics. If you can accept working with GNU extensions that are available in recent-ish GCC and Clang (but not MSVC, not sure about Intel ICC), there are pretty nice vector extensions [0]. With them you can get standard binary operators working for arithmetic (+,-,*,/ etc) and shuffling with __builtin_shuffle. These are CPU independent, the sam…

> If you can accept working with GNU extensions that are available in recent-ish GCC and Clang I do my private project in C++ so it's not a case, but at my current company we use also MSVC. I wish we could abandon that compiler and work with GCC or clang only. > However, for a lot of SIMD tasks I encounter, just basic arithmetic + shuffles is more than 80% of what I need. Your remaining 20% is my 80%. :)

> ... but at my current company we use also MSVC. I wish we could abandon that compiler and work with GCC or clang only.

Good news! These days you can produce MSVC compatible binaries with Clang or even use Clang as a compiler from the C++ IDE.

Whether or not you can do this in practice is another matter, but it can be done.

> Your remaining 20% is my 80%. :)

Yeah, if you look at my examples, they're rather straightforward arithmetic with 4 dimensional vectors. There's very little need for any integer arithmetic or more exotic combinations of operations. A little fused multiply-and-add here and there.

But I haven't seen a better method for this, most of the code is CPU-agnostic and will compile to x86 or ARM code using all the available instruction sets (depending on compiler arguments, e.g. -mavx2 or -march=native). I really haven't seen a SIMD math lib with so little duplication for different CPUs elsewhere.

Re: SSE: mind the gap

#35

Post by nivertech »

johnt15 wrote: post #2

It's much better to use any of the numerous SIMD wrappers such as libsimdpp or Vc and get various benefits for free. It's possible to target everything from SSE and NEON to AVX512 with what is essentially a single code path.

Why to use wrapper libraries when you can use OpenCL for CPU compute device?
Post reply on HN