Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SSE: mind the gap

fgiesen.wordpress.com

Post reply on HN
Print view View original

11–20 of 35 posts

Re: SSE: mind the gap

#11

Post by wmu »

johnt15 wrote: post #7
wmu wrote: post #4

Earlier quoted context omitted.

Not sure about "single code path". Differences amid SIMD flavors are significant, there are cases when translation one-to-one is either impossible or unpractical. A bright example are AVX2 instructions operating on 128-bit lanes rather whole 256-bit registers. And wrappers exists in the C++ ecosystem, C programmers are stuck to intrinsics.

The property of AVX and AVX2 you mentioned actually helps having single code path. If the SIMD wrapper allows parameterization on vector width (most do that), you can simply increase vector width when compiling for AVX and that's it.

I understand you point, however it not as simple as it seems. Of course, for trivial code transition between different SIMD flavors could be seamless. But the world is cruel. :)

Think about shuffling instructions (pshufb), lookup vector for the instruction are different in AVX2 and SSE. Even if an AVX2 vector could be created by cloning SSE vector twice, this must be a programmer decision.

Another example is algorithm using video-encoding instruction mpsadbw to locate substrings (http://0x80.pl/articles/sse4_substring_locate.html#introduct...). AVX2 instruction vmpsadw operates on 128-bit lanes and the algorithm have to be rewritten in some parts to align with this limitation.

Re: SSE: mind the gap

#12

Post by cm3 »

I'm trying to understand speculative execution.

Given this

    int result = foo != bar ? do_side_effect_and_return() : safe_return();
C code, am I right to assume both functions will be executed speculatively?

What other potential bugs/gotchas are lurking with speculative execution?

Re: SSE: mind the gap

#13

Post by daemin »

cm3 wrote: post #12

I'm trying to understand speculative execution. Given this int result = foo != bar ? do_side_effect_and_return() : safe_return(); C code, am I right to assume both functions will be executed speculatively? What other potential bugs/gotchas are lurking with speculative execution?

In the context of a CPU, especially one with a deeper pipeline, the comparison value will only be known at some stage deep within the pipeline. Therefore to not stall the pipeline until the result is known, the CPU will start to execute either one of the branches. Then once the value is known, if it guessed the branch correctly it will continue executing as normal, having already partially executed it. But if it guesses incorrectly it has to flush the work that it has done and start executing the branch not taken.

Edit: One particular thing to note is that side effects that occur within the pipeline do not actually occur until the latter stages of the pipeline, where the writes to memory and registers are realised. By that stage the condition result would already be known and the correct branch would be executing within the processor.

Re: SSE: mind the gap

#14

Post by cm3 »

daemin wrote: post #13
cm3 wrote: post #12

I'm trying to understand speculative execution. Given this int result = foo != bar ? do_side_effect_and_return() : safe_return(); C code, am I right to assume both functions will be executed speculatively? What other potential bugs/gotchas are lurking with speculative execution?

In the context of a CPU, especially one with a deeper pipeline, the comparison value will only be known at some stage deep within the pipeline. Therefore to not stall the pipeline until the result is known, the CPU will start to execute either one of the branches. Then once the value is known, if it guessed the branch correctly it will continue executing as normal, having already partially executed it. But if it gues…

Yeah, but what if do_side_effect_and_return() deletes file. This surely cannot be prevented.

What I'm mostly wondering is how it is that half of our code doesn't break all the time due to speculative execution.

I'm looking for an explanation how it's prevented or if it's just careful luck because compiler and runtime writers took precautions.

Re: SSE: mind the gap

#15

Post by cm3 »

clevernickname wrote: post #10
johnt15 wrote: post #2

It's much better to use any of the numerous SIMD wrappers such as libsimdpp or Vc and get various benefits for free. It's possible to target everything from SSE and NEON to AVX512 with what is essentially a single code path.

Realistically the vast majority of C and C++ codebases today will never touch anything more than x86 and ARM, and I wouldn't be surprised if most never even get past x86, so I don't buy the portability argument. Portability between SSE and AVX is a better argument. But in any case, if you're using SIMD in anger, chances are you have hard performance requirements that you really care about, and a one size fits all app…

> Realistically the vast majority of C and C++ codebases today will never touch anything more than x86 and ARM, and I wouldn't be surprised if most never even get past x86, so I don't buy the portability argument.

Just recently a Gentoo developer ported GHC to m68k and found some portability issues who fixed in the process, which benefit all architectures. This is also why OpenBSD devs are still on gcc3.

RISC and POWER are just two very modern ISAs to mention and not something you can ignore easily. We need more ISAs like in the past, not just two. It's very dangerous to limit ourselves to just ARM/x86 and diversity is a plus for writing more correct code and having more options. lowRISC is a nice fit for many things as is POWER, while of course ARM and x86 are here to stay. I'd count Nvidia's and AMD's GPUs as the other major architectures, but we don't usually deal directly at that level with GPUs. You choose the right chip for the job, just as phones select different SoCs for different use cases.

Re: SSE: mind the gap

#16

Post by exDM69 »

wmu wrote: post #4
johnt15 wrote: post #2

It's much better to use any of the numerous SIMD wrappers such as libsimdpp or Vc and get various benefits for free. It's possible to target everything from SSE and NEON to AVX512 with what is essentially a single code path.

Not sure about "single code path". Differences amid SIMD flavors are significant, there are cases when translation one-to-one is either impossible or unpractical. A bright example are AVX2 instructions operating on 128-bit lanes rather whole 256-bit registers. And wrappers exists in the C++ ecosystem, C programmers are stuck to intrinsics.

> And wrappers exists in the C++ ecosystem, C programmers are stuck to intrinsics.

If you can accept working with GNU extensions that are available in recent-ish GCC and Clang (but not MSVC, not sure about Intel ICC), there are pretty nice vector extensions [0].

With them you can get standard binary operators working for arithmetic (+,-,*,/ etc) and shuffling with __builtin_shuffle. These are CPU independent, the same code compiles neatly to ARM NEON as well as x86 SSE+AVX+FMA. All you need is a typedef with an __attribute__.

The vector extension functions don't cover the whole instruction sets but the vector types are compatible with _mm128 and NEON native formats so you can resort to intrinsics when necessary.

However, for a lot of SIMD tasks I encounter, just basic arithmetic + shuffles is more than 80% of what I need.

If you want to see some examples, take a look at my collection of 3d graphics and physics related SIMD routines [1]. (note: this project could use some help, let me know if you're interested in doing something with it or porting some of the hand optimized routines to more used math libs like glm)

[0] https://gcc.gnu.org/onlinedocs/gcc/Vector-Extensions.html#Ve... [1] https://github.com/rikusalminen/threedee-simd

Re: SSE: mind the gap

#17

Post by panic »

cm3 wrote: post #14
daemin wrote: post #13

Earlier quoted context omitted.

In the context of a CPU, especially one with a deeper pipeline, the comparison value will only be known at some stage deep within the pipeline. Therefore to not stall the pipeline until the result is known, the CPU will start to execute either one of the branches. Then once the value is known, if it guessed the branch correctly it will continue executing as normal, having already partially executed it. But if it gues…

Yeah, but what if do_side_effect_and_return() deletes file. This surely cannot be prevented. What I'm mostly wondering is how it is that half of our code doesn't break all the time due to speculative execution. I'm looking for an explanation how it's prevented or if it's just careful luck because compiler and runtime writers took precautions.

The CPU won't speculatively execute past certain instructions. AFAIK it's up to (e.g.) the SSD driver to prevent the CPU from speculatively sending commands to the drive.

Re: SSE: mind the gap

#18

Post by gliptic »

panic wrote: post #17
cm3 wrote: post #14

Earlier quoted context omitted.

Yeah, but what if do_side_effect_and_return() deletes file. This surely cannot be prevented. What I'm mostly wondering is how it is that half of our code doesn't break all the time due to speculative execution. I'm looking for an explanation how it's prevented or if it's just careful luck because compiler and runtime writers took precautions.

The CPU won't speculatively execute past certain instructions. AFAIK it's up to (e.g.) the SSD driver to prevent the CPU from speculatively sending commands to the drive.

It's not up to the SSD driver. The SSD driver will not see any effects of speculatively executed code and there's no way the CPU could speculatively send commands to the drive.

Re: SSE: mind the gap

#19

Post by panic »

gliptic wrote: post #18
panic wrote: post #17

Earlier quoted context omitted.

The CPU won't speculatively execute past certain instructions. AFAIK it's up to (e.g.) the SSD driver to prevent the CPU from speculatively sending commands to the drive.

It's not up to the SSD driver. The SSD driver will not see any effects of speculatively executed code and there's no way the CPU could speculatively send commands to the drive.

What if you're communicating with the SSD via DMA? Don't you have to make sure that the writes aren't speculatively executed?

Re: SSE: mind the gap

#20

Post by daemin »

cm3 wrote: post #14
daemin wrote: post #13

Earlier quoted context omitted.

In the context of a CPU, especially one with a deeper pipeline, the comparison value will only be known at some stage deep within the pipeline. Therefore to not stall the pipeline until the result is known, the CPU will start to execute either one of the branches. Then once the value is known, if it guessed the branch correctly it will continue executing as normal, having already partially executed it. But if it gues…

Yeah, but what if do_side_effect_and_return() deletes file. This surely cannot be prevented. What I'm mostly wondering is how it is that half of our code doesn't break all the time due to speculative execution. I'm looking for an explanation how it's prevented or if it's just careful luck because compiler and runtime writers took precautions.

Ok, from what I see you are thinking at too high a level for this.

The CPU has a pipeline where it executes instructions. This pipeline has stages in it for: fetching the instruction from L1 cache (a), decoding the instruction (b), fetching data from registers or memory (c), computing the instruction (d), storing the data back into registers or memory (e). This is a rough grouping of these stages, and each of these (a-e) stages is comprised of smaller stages, one of which executes in each clock cycle.

So between the first stage where a CPU starts executing a comparison instruction (a), and when it knows what the value is (d) that can be many clock cycles. So instead of waiting and stalling it instead guesses which branch will be taken and starts feeding it in.

It is only in stage (e) where it stores values to registers or memory where actions can actually take place, and by the time it gets there it is executing the correct branch. Either because it guessed the correct branch from the beginning or it has mispredicted, flushed the pipeline, and is now executing the correct branch.

Edit: Note that there is no such instruction as "delete file on SSD". The CPU has various ways of working with external devices (such as sounds/video chips, ssd's, etc), with memory mapping being one of the more popular one, but there's also IO pins and a variety of hardware protocols that it can write instructions to use. If you want to learn up on this get a small device like a Raspberry Pi and play around.

Post reply on HN