Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A bug that doesn’t exist on x86: Exploiting an ARM-only race condition

github.com

Post reply on HN
Print view View original

111–120 of 141 posts

Re: A bug that doesn’t exist on x86: Exploiting an ARM-only race condition

#111

Post by [deleted] »

vlovich123 wrote: post #98
kloch wrote: post #84

Earlier quoted context omitted.

I did tests on Kahan summation recently on my macbook pro and -O3 defeated the algorithm while -O2 did not. Declaring the below variables as volatile restored error compensation with -O3. The relevant code is: kahan_y=g_sample_z - kahan_c; kahan_t=g_sample_z_sum + kahan_y; kahan_c=(kahan_t - g_sample_z_sum) - kahan_y; g_sample_z_sum=kahan_t; (this is in an inner loop where a new g_sample_z is calculated and then adde…

Sounds like a compiler bug to me. Can you file a bug to clang with a reduced standalone test (or I can do it for you if you share the standalone test).

[deleted]

Re: A bug that doesn’t exist on x86: Exploiting an ARM-only race condition

#112

Post by kloch »

vlovich123 wrote: post #98
kloch wrote: post #84

Earlier quoted context omitted.

I did tests on Kahan summation recently on my macbook pro and -O3 defeated the algorithm while -O2 did not. Declaring the below variables as volatile restored error compensation with -O3. The relevant code is: kahan_y=g_sample_z - kahan_c; kahan_t=g_sample_z_sum + kahan_y; kahan_c=(kahan_t - g_sample_z_sum) - kahan_y; g_sample_z_sum=kahan_t; (this is in an inner loop where a new g_sample_z is calculated and then adde…

Sounds like a compiler bug to me. Can you file a bug to clang with a reduced standalone test (or I can do it for you if you share the standalone test).

Here is a complete simplified Kahan summation test and indeed it works with -O3 but fails with -Ofast. There must have been something else going on in my real program at -O3. However the original point that 'volatile' can be a workaround for some optimization problems is still valid (you may want the rest of your program to benefit from -Ofast without breaking certain parts).

Changing the three kahan_* variables to volatile makes this work (slowly) with -Ofast.

  #include 

  int main(int argc, char **argv) {
    int i;
    double sample, sum;
    double kahan_y, kahan_t, kahan_c;

    // initial values
    sum=0.0;
    sample=1.0; // start with "large" value

    for (i=0; i

Re: A bug that doesn’t exist on x86: Exploiting an ARM-only race condition

#113

Post by ddlutz »

aydwi wrote: post #107
vitus wrote: post #40

I spent some time trying to figure out why the lock-free read/write implementation is correct under x86, assuming a multiprocessor environment. My read of the situation was that there's already potential for a double-read / double-write between when the spinlock returns and when the head/tail index is updated. Turns out that I was missing something: there's only one producer thread, and only one consumer thread. If t…

> IMO the use of `new` in modern C++ (as is the case in the writer queue) is often a code smell As a naive practitioner of modern C++, I'd love it if you could elaborate on this.

It's widely accepted you should almost always use some smart pointer like unique_ptr or shared_ptr, and even sometimes the the unpopular weak_ptr depending on your use case.

Re: A bug that doesn’t exist on x86: Exploiting an ARM-only race condition

#114

Post by oriki »

codeflo wrote: post #67
cesarb wrote: post #57

Earlier quoted context omitted.

Unless you're the maintainer of the parking_lot crate, you're not "inventing your own". And since parking_lot is AFAIK the second most popular implementation of mutexes and RW locks in Rust (the most popular one being obviously the one in the Rust standard library, which wraps the OS-provided lock implementations), you can assume it's well tested.

Everything about which people tell you to “not invent your own” must be invented by someone .

I mean, that's too literal of an interpretation.

It's more like "if you don't need to, don't invent your own [x]." People who like to invent [x] are usually smart enough to understand why that warning is there to begin with, and don't tend to argue with it.

Re: A bug that doesn’t exist on x86: Exploiting an ARM-only race condition

#115

Post by rsynnott »

SavantIdiot wrote: post #105
stjohnswarts wrote: post #61

Earlier quoted context omitted.

I doubt that. The number of ARM processors is far greater in reality than in x86 if we clarify it by saying “in operation” rather than historically and these stories will become more common but certainly won't see a “sharp increase”.

The dominant Arm core in the world is a Cortex-M (or Cortex-R) which are single-core. They are 99% of the time on a die with far less These outnumber x86+Cortex-A by probably a factor of 1,000.

There are almost certainly more multi-core ARM chips than x86 chips around, too, tho.

Re: A bug that doesn’t exist on x86: Exploiting an ARM-only race condition

#116

Post by SavantIdiot »

rsynnott wrote: post #115
SavantIdiot wrote: post #105

Earlier quoted context omitted.

The dominant Arm core in the world is a Cortex-M (or Cortex-R) which are single-core. They are 99% of the time on a die with far less These outnumber x86+Cortex-A by probably a factor of 1,000.

There are almost certainly more multi-core ARM chips than x86 chips around, too, tho.

That's a good question. I guess we would need to compare all the arm multi-core in smartphones & some laptops, vs all the intel laptops/desktops/servers. Hmmm... tough one.

Re: A bug that doesn’t exist on x86: Exploiting an ARM-only race condition

#117

Post by addaon »

saagarjha wrote: post #38
amelius wrote: post #36

Does the race condition exist when emulating x86 on Apple M1?

No. Rosetta emulates TSO correctly.

To draw together the two answers here to the original question.

1) Emulating an ISA includes emulating its memory model. As saagarjha says, this means that Rosetta 2 must (and does) correctly implement total store ordering.

2) There are various ways to implement this. For emulators that include a binary translation layer (that is, that translate x86 opcodes into a sequence of ARM opcodes), one route is to generate the appropriate ARM memory barriers as part of the translation. Even with optimization to reduce the number of necessary barriers, though, this is expensive. Instead, as mmwelt mentions, Apple took an unusual route here. The Apple Silicon MMU can be configured on a per-page basis to use either the relaxed ARM memory model or the TSO x86 memory model. There is a performance cost at the hardware level for using TSO, and there is a cost in silicon area for supporting both; but from the point of view of Rosetta 2, all it has to do is mark x86-accessed pages as TSO and the hardware takes care of the details, no software memory barriers needed.

Re: A bug that doesn’t exist on x86: Exploiting an ARM-only race condition

#118

Post by vlovich123 »

kloch wrote: post #112
vlovich123 wrote: post #98

Earlier quoted context omitted.

Sounds like a compiler bug to me. Can you file a bug to clang with a reduced standalone test (or I can do it for you if you share the standalone test).

Here is a complete simplified Kahan summation test and indeed it works with -O3 but fails with -Ofast. There must have been something else going on in my real program at -O3. However the original point that 'volatile' can be a workaround for some optimization problems is still valid (you may want the rest of your program to benefit from -Ofast without breaking certain parts). Changing the three kahan_* variables to v…

Correct. `-Ofast` claim to fame is it enables `-ffast-math` which is why it has huge warning signs around it in the documentation. `-ffast-math` turns on associativity which is problematic for Kahan summation. Rather than sprinkling in volatiles which pessimizes the compiler to no end, I would recommend annotating the problematic function to turn off associativity [1][2].

Something like:

    [[gnu::optimize("no-associative-math")]]
    double kahanSummation() {
      ...
    }
That way the compiler applies all the optimizations it can but only turns off associative math. This should work on Clang & GCC & be net faster in all cases.

This is what I mean by "If you're sprinkling volatile around, you probably aren't doing what you want" and are just cargo culting bad advice.

[1] https://stackoverflow.com/questions/26266820/in-clang-how-do... [2] https://gcc.gnu.org/onlinedocs/gcc-4.7.0/gcc/Function-Attrib...

Re: A bug that doesn’t exist on x86: Exploiting an ARM-only race condition

#119

Post by monocasa »

xxs wrote: post #34
monocasa wrote: post #18

Earlier quoted context omitted.

And since the compiler omitted it on x86, an x86 emulator doesn't have access to where they're required as seen by the compiler.

emulator would have a zero issue, if it's a direct transfer for assembly (not an emulator), it'd need either hardware support - e.g. apple chips, or memory barriers. The differences between arm and x86 are known for 15y+, there is nothing new about it. Also concurrency support is one of the major benefits of languages with proper memory model - java started it with JMM[0] [0]: https://www.cs.umd.edu/~pugh/java/memory…

The differences are very known, it's just still an open problem how to address running code for a stronger memory model on systems with a weaker one at as high as possible performance performance without explicit hardware support (like Apple's choice of a TSO config bit). Your compiled binary for TSO has erased any LoadLoad, LoadStore, and StoreStore barriers, and the emulator has to divine them. The heuristics there are still fraught with peril.

The JVM absolutely did some great work walking this path, both in defining a memory model in the first place, and supporting that model on weak and strong hardware memory models, but the JMM was specifically designed to be able to run cleanly on WMO platforms to begin with (early Sparc), so they don't face a lot of the same problems discussed here.

Re: A bug that doesn’t exist on x86: Exploiting an ARM-only race condition

#120

Post by saagarjha »

addaon wrote: post #117
saagarjha wrote: post #38

Earlier quoted context omitted.

No. Rosetta emulates TSO correctly.

To draw together the two answers here to the original question. 1) Emulating an ISA includes emulating its memory model. As saagarjha says, this means that Rosetta 2 must (and does) correctly implement total store ordering. 2) There are various ways to implement this. For emulators that include a binary translation layer (that is, that translate x86 opcodes into a sequence of ARM opcodes), one route is to generate th…

TSO is a per-thread flag; it’s implemented as a write to a MSR on context switch to switch it on or off.
Post reply on HN