SSE: mind the gap • Hacker News Board

Re: SSE: mind the gap

#21

Post by daemin » Mon, Apr 04, 2016, 11:09 AM UTC

panic wrote: post #19

gliptic wrote: post #18

Earlier quoted context omitted.

It's not up to the SSD driver. The SSD driver will not see any effects of speculatively executed code and there's no way the CPU could speculatively send commands to the drive.

What if you're communicating with the SSD via DMA? Don't you have to make sure that the writes aren't speculatively executed?

Because DMA occurs when the CPU writes to memory, and as my comment above states this occurs after the condition is evaluated and the jump instruction (not) taken.

Speculative execution is purely within the CPU and doesn't leak out - except in the case of worse than ideal performance.

Re: SSE: mind the gap

#22

Post by clevernickname » Mon, Apr 04, 2016, 11:42 AM UTC

cm3 wrote: post #15

clevernickname wrote: post #10

Earlier quoted context omitted.

Realistically the vast majority of C and C++ codebases today will never touch anything more than x86 and ARM, and I wouldn't be surprised if most never even get past x86, so I don't buy the portability argument. Portability between SSE and AVX is a better argument. But in any case, if you're using SIMD in anger, chances are you have hard performance requirements that you really care about, and a one size fits all app…

> Realistically the vast majority of C and C++ codebases today will never touch anything more than x86 and ARM, and I wouldn't be surprised if most never even get past x86, so I don't buy the portability argument. Just recently a Gentoo developer ported GHC to m68k and found some portability issues who fixed in the process, which benefit all architectures. This is also why OpenBSD devs are still on gcc3. RISC and POW…

The idea that compiling your code for 68000 or MIPS can reveal bugs in your code does not change the fact that x86 and ARM are pretty much the only relevant CPU architectures that all but the most entrenched of government contractors could ship a product on today or in the foreseeable future that would have any use for SIMD. If you actually have a need to do extensive SIMD optimizations (say, it could shave 5ms off your frame time in a game, or save you $XXXXXX/year in your data center), PowerPC does not enter your mind at any moment.

You see it as weeding out bugs and future proofing your code in case x86 or ARM disappears tomorrow, I see it as a load of completely wasted work and optimization opportunities.

Also lowRISC learned nearly nothing from the past 20 years of CPU architecture advancement. It is not modern, it is a naive copy of a very outdated design.

Re: SSE: mind the gap

#23

Post by Ono-Sendai » Mon, Apr 04, 2016, 12:45 PM UTC

johnm1019 wrote: post #9

andrewf wrote: post #3

Earlier quoted context omitted.

Would you be able to point me towards a shipping product/library that does this? It's easy to find examples of people hardcoding x64 assembly (x264, zlib, libyuv) but I haven't stumbled across anybody making good use of a high level wrapper.

Although it's way more than an SSE wrapper, the Eigen library is excellent in my experience and targets multiple platforms. http://eigen.tuxfamily.org/index.php?title=Main_Page

I had a look at the matrix*vector multiplication code for Eigen once and it was rubbish.

Re: SSE: mind the gap

#24

Post by johnt15 » Mon, Apr 04, 2016, 1:40 PM UTC

clevernickname wrote: post #10

johnt15 wrote: post #2

It's much better to use any of the numerous SIMD wrappers such as libsimdpp or Vc and get various benefits for free. It's possible to target everything from SSE and NEON to AVX512 with what is essentially a single code path.

Realistically the vast majority of C and C++ codebases today will never touch anything more than x86 and ARM, and I wouldn't be surprised if most never even get past x86, so I don't buy the portability argument. Portability between SSE and AVX is a better argument. But in any case, if you're using SIMD in anger, chances are you have hard performance requirements that you really care about, and a one size fits all app…

By saying single code path, I don't mean single instruction stream. libsimdpp, for example, supports building same code for different instruction sets, linking into the same executable and then dispatching dynamically. Doing this by hand would mean that either:

- lots of time is wasted creating slightly different versions of code. I'm talking about e.g. AVX vs. AVX2 for floating-point code not SSE2 vs. AVX.

- micro-optimization opportunities are wasted by only coding for major revisions of the instruction set

Even when optimal performance may only be achieved via completely different approaches, the SIMD wrappers are easier to use, because they present consistent interface. Any specialized instructions may be used by simply falling back to native intrinsics.

Thus I don't see much benefit of writing SIMD code without a wrapper. The only advantage is that it's harder to shoot oneself into the foot with naive use of these wrappers, e.g. if one doesn't actually look into the generated assembly code.

Re: SSE: mind the gap

#25

Post by pjc50 » Mon, Apr 04, 2016, 2:30 PM UTC

cm3 wrote: post #12

I'm trying to understand speculative execution. Given this int result = foo != bar ? do_side_effect_and_return() : safe_return(); C code, am I right to assume both functions will be executed speculatively? What other potential bugs/gotchas are lurking with speculative execution?

Firstly, the article doesn't mention speculative execution - it refers to a programming technique which is effectively (assume f,g,h are arithmetic functions, not 'program' functions):

  compute f
  while f is in the pipeline, compute g
  while g is in the pipeline, compute h
  let bits = 111111..111 if h is true or 00000...000 if h is false
  let result = (f & bits) | (g & ~bits)

In your example, if they're function calls, and not inlined, then they won't execute speculatively at all: speculative execution usually only applies to straight-line instructions and in any case only applies to letting the instruction go off into an "execution" unit. Only one branch should ever make it into the "commit" phase of the pipeline (think like a database commit), and only one should ever have effects that are visible off the processor die.

Also things like system calls, interrupts, context switches, and so on tend to flush the pipeline and insert a "barrier", at which point all the instructions before the barrier have committed and none of the ones after the barrier have committed.

Re: SSE: mind the gap

#26

Post by cm3 » Mon, Apr 04, 2016, 7:43 PM UTC

daemin wrote: post #20

cm3 wrote: post #14

Earlier quoted context omitted.

Yeah, but what if do_side_effect_and_return() deletes file. This surely cannot be prevented. What I'm mostly wondering is how it is that half of our code doesn't break all the time due to speculative execution. I'm looking for an explanation how it's prevented or if it's just careful luck because compiler and runtime writers took precautions.

Ok, from what I see you are thinking at too high a level for this. The CPU has a pipeline where it executes instructions. This pipeline has stages in it for: fetching the instruction from L1 cache (a), decoding the instruction (b), fetching data from registers or memory (c), computing the instruction (d), storing the data back into registers or memory (e). This is a rough grouping of these stages, and each of these (…

I see, are CLWB and PCOMMIT for NVME safe?

Re: SSE: mind the gap

#27

Post by vardump » Mon, Apr 04, 2016, 9:47 PM UTC

cm3 wrote: post #26

daemin wrote: post #20

Earlier quoted context omitted.

Ok, from what I see you are thinking at too high a level for this. The CPU has a pipeline where it executes instructions. This pipeline has stages in it for: fetching the instruction from L1 cache (a), decoding the instruction (b), fetching data from registers or memory (c), computing the instruction (d), storing the data back into registers or memory (e). This is a rough grouping of these stages, and each of these (…

I see, are CLWB and PCOMMIT for NVME safe?

Yes. They're no exception. They actually make DMA transfer safe, without doing unnecessary work.

Re: SSE: mind the gap

#28

Post by cm3 » Mon, Apr 04, 2016, 9:57 PM UTC

vardump wrote: post #27

cm3 wrote: post #26

Earlier quoted context omitted.

I see, are CLWB and PCOMMIT for NVME safe?

Yes. They're no exception. They actually make DMA transfer safe, without doing unnecessary work.

So all this speculative business is more about using idle bits of the chip to warm it up for either branch to be taken, thereby reducing some of the time for that code to execute, but any and all op that would actually modify memory or access stuff on the bus is exempt from speculation.

About right?

Re: SSE: mind the gap

#29

Post by clevernickname » Mon, Apr 04, 2016, 10:13 PM UTC

johnt15 wrote: post #24

clevernickname wrote: post #10

Earlier quoted context omitted.

Realistically the vast majority of C and C++ codebases today will never touch anything more than x86 and ARM, and I wouldn't be surprised if most never even get past x86, so I don't buy the portability argument. Portability between SSE and AVX is a better argument. But in any case, if you're using SIMD in anger, chances are you have hard performance requirements that you really care about, and a one size fits all app…

By saying single code path, I don't mean single instruction stream. libsimdpp, for example, supports building same code for different instruction sets, linking into the same executable and then dispatching dynamically. Doing this by hand would mean that either: - lots of time is wasted creating slightly different versions of code. I'm talking about e.g. AVX vs. AVX2 for floating-point code not SSE2 vs. AVX. - micro-o…

Yeah, I understood what you meant, I've used wrappers like that before. My contention was with your original comment,

>It's possible to target everything from SSE and NEON to AVX512 with what is essentially a single code path.

the practice of which does not generally make the best usage of any particular instruction set, emulating certain operations that aren't available on a platform with multiple instructions, etc. It might be good enough for many light optimization jobs, in which case I'd say go for it, you're doing so much better than the vast majority of programmers writing Python or whatever. But what I was trying to argue was that if you really need to crunch the hell out of some numbers, then you probably have a small set of target platforms that you can justify directly using intrinsics (or even assembly) for.

This claim, however:

>I'm talking about e.g. AVX vs. AVX2 for floating-point code not SSE2 vs. AVX.

is a lot more reasonable, but you could do the same with some strategically placed #ifdefs with native intrinsics or assembly.

Re: SSE: mind the gap

#30

Post by vardump » Mon, Apr 04, 2016, 11:04 PM UTC

cm3 wrote: post #28

vardump wrote: post #27

Earlier quoted context omitted.

Yes. They're no exception. They actually make DMA transfer safe, without doing unnecessary work.

So all this speculative business is more about using idle bits of the chip to warm it up for either branch to be taken, thereby reducing some of the time for that code to execute, but any and all op that would actually modify memory or access stuff on the bus is exempt from speculation. About right?

> About right?

Close.

There are two kinds of speculative execution.

1) CPU guesses (branch predicts) one path by default. If it guesses wrong, it'll need to throw away speculative results and to execute the alternative path. No speculative state is leaked to other CPU cores or memory (writes or I/O). Typically CPUs guess right way over 99% of time -- there are of course cases when prediction fails, sometimes pathologically. Those times it guesses wrong, 15-20 cycles are lost. To put it in perspective, that's enough cycles for 500 floating point computations.

2) Programmer / compiler produced speculative execution. Typical for SIMD, for both CPUs and GPUs. For example with AVX2 you could compute results for 16x 16 bit integer lanes (256 bits wide) per instruction (so maybe about 32x 16-bit operations per clock cycle). Computations for both branches are done in parallel and right results are masked before writing data somewhere. The benefit is ability to avoid branching, gaining performance.

Sometimes CPU branch predictor does really badly. For example, if you have somewhat random data dependant branching, CPU is going to guess wrong 50% time. So computing both sides in parallel and throwing 50% of the results away might mean an order of magnitude speedup!