Live data from Hacker News

Ways to break your systems code using volatile (2010)

blog.regehr.org

51–60 of 77 posts

Re: Ways to break your systems code using volatile (2010)

#51

Ironically volatile is just as bad in Java for different reasons. Frequently used for "lock free" synchronization, its usually actually worse than using locks because it can't be cached between cores. The variable is always loaded from main memory, which is usually much worse than holding a lock mutex in registers.

> Frequently used for "lock free" synchronization, its usually actually worse than using locks If you want lock-free what do you suggest we use instead of volatile? > which is usually much worse than holding a lock mutex in registers How can you hold a mutex in a register? That doesn't make any sense.

You can use a lock but held in a regular CPU register. They're just regular variables for the most part.

Lock free in Java is usually worse that what the JVM can pull off with lock elison

Re: Ways to break your systems code using volatile (2010)

#52
post #5

Ironically volatile is just as bad in Java for different reasons. Frequently used for "lock free" synchronization, its usually actually worse than using locks because it can't be cached between cores. The variable is always loaded from main memory, which is usually much worse than holding a lock mutex in registers.

The standard pattern for working with atomics in java (volatiles are of limited use without atomic field updaters or varhandles) is to read it into a local variable, operate on that and only write it back to the volatile once you're done. That has many benefits, among them the ability to store its value in registers.

For primitives Java uses special CPU instructions. In the Atomic* package. It's not recommended for plain objects.

Re: Ways to break your systems code using volatile (2010)

#53
post #8

volatile should only be used for accessing MMIO registers in device drivers; that's it.

There's definitely more uses. For example, shared memory between processes: you should mark it volatile. C++ atomics are no good here, because they are not guaranteed to be lock free or address free.

Shared memory is way outside the scope of standard C or C++. It's implementation-defined. It's inconsistent to insist on the weakest definition of atomics allowed by the C/C++ standard(s) and simultaneous invoke one of the weirdest implementation-defined mechanisms defined by POSIX. If your implementation provides shared memory of some kind, it's up to your implementation to define some sort of reasonable semantics.

In POSIX' case, it's up to POSIX operating systems to define reasonable semantics on the memory, using constructs like PTHREAD_PROCESS_SHARED and "robust" pthread mutexes.

Re: Ways to break your systems code using volatile (2010)

#54

Earlier quoted context omitted.

> Frequently used for "lock free" synchronization, its usually actually worse than using locks If you want lock-free what do you suggest we use instead of volatile? > which is usually much worse than holding a lock mutex in registers How can you hold a mutex in a register? That doesn't make any sense.

You can use a lock but held in a regular CPU register. They're just regular variables for the most part. Lock free in Java is usually worse that what the JVM can pull off with lock elison

> You can use a lock but held in a regular CPU register. They're just regular variables for the most part.

I don't understand this. If your lock variable is in a CPU register how do other CPUs acquire the lock?

> Lock free in Java is usually worse that what the JVM can pull off with lock elision

I don't understand this either. Java's lock elision is only going to make a concurrent object 'lock-free' in the case where the object does not escape the compilation unit. In which case again how would another thread use it? Java will also combine adjacent critical sections created by monitors even if they escape, but it won't make them lock-free in that case.

Re: Ways to break your systems code using volatile (2010)

#55

Earlier quoted context omitted.

You can use a lock but held in a regular CPU register. They're just regular variables for the most part. Lock free in Java is usually worse that what the JVM can pull off with lock elison

> You can use a lock but held in a regular CPU register. They're just regular variables for the most part. I don't understand this. If your lock variable is in a CPU register how do other CPUs acquire the lock? > Lock free in Java is usually worse that what the JVM can pull off with lock elision I don't understand this either. Java's lock elision is only going to make a concurrent object 'lock-free' in the case where…

JVM is very smart about locking. My knowledge is limited but this is a great article. https://shipilev.net/jvm/anatomy-quarks/19-lock-elision/

Re: Ways to break your systems code using volatile (2010)

#56
post #20

Earlier quoted context omitted.

Could you clarify? I am merely a student of that class and we didn't discuss TLBs in detail so I'm all ears for details.

On an architecture with virtual protected memory (the one being described in the slides) there is no compiler control over the TLB. There is no mechanism for the compiler to bypass it. It isn't the semantics in theory or in practice for volatile to bypass almost anything on that list you have for the non-volatile case. It just isn't true. There must be some misunderstanding somewhere that is only clarified viva voce.…

[deleted]

Re: Ways to break your systems code using volatile (2010)

#57
post #8

volatile should only be used for accessing MMIO registers in device drivers; that's it.

That is a reasonable heuristic but your statement is not technically correct. E.g. you need volatile around setjmp/longjmp and that has nothing to do with IO.

Re: Ways to break your systems code using volatile (2010)

#58
post #28

Earlier quoted context omitted.

Yep. The slide is completely wrong. It is showing low-level architecture details that would be 100% identical between the two cases. Volatile changes nothing on that list. Volatile just makes sure the compiler bothers. Otherwise, a pair of writes to the same memory location could be optimized by eliminating the first write. Volatile makes the compiler do that. Of course, the CPU itself may then do this optimization,…

> It is showing low-level architecture details that would be 100% identical between the two cases. To be as charitable as I can possibly be, the only part that could theoretically make sense is that the compiler could emit non-temporal store instructions to bypass the cache. I know compilers currently don't do that for volatile, but I don't know why.

> the only part that could theoretically make sense is that the compiler could emit non-temporal store instructions to bypass the cache. I know compilers currently don't do that for volatile, but I don't know why.

Two reasons:

First, using nontemporal accesses would break mixed volatile and non-volatile accesses to the same memory, something which is not defined by the C standard but which some programs rely on anyway.

Second, more importantly: why would they?

- If the address you’re accessing points to hardware registers, the page table entry should be marked non-cacheable, which makes nontemporal accesses unnecessary. And if for some reason it’s not marked properly, nontemporal accesses wouldn’t be sufficient to guarantee that things work anyway, because nontemporal is just a hint which the hardware may not respect. In any case, at least on x86, AFAIK the only nontemporal instructions access 128+ bits of memory at a time, which wouldn’t even work for hardware registers (which generally require you to use a specific access size).

- If the address you’re using points to regular memory, on the other hand, volatile is probably being used to implement atomics, in which case bypassing the cache is unnecessary and also slow. In theory, compilers could compile volatile into accesses surrounded by memory barrier instructions, which would enforce a stronger memory ordering (while being faster than bypassing the cache entirely), especially useful on architectures with weaker memory models than x86. In fact, that’s what volatile does in Java. But in C, it’s pretty long-established that volatile accesses should just compile to regular load/store instructions, and any necessary barriers must be inserted manually. People writing high-performance code wouldn’t be happy if the compiler started inserting unnecessary barrier instructions for them… In any case, usage of volatile for atomics is deprecated in favor of C/C++11 atomics, which do insert barriers for you.

Re: Ways to break your systems code using volatile (2010)

#59
post #8

volatile should only be used for accessing MMIO registers in device drivers; that's it.

This is probably not useful for production, but volatile is a great way to see what kind of code compiler generates in a realistic setting. For example, if you want to see how compiler optimizes a code snippet and the code depends on a constant that you don't want to get constant folded away.

Re: Ways to break your systems code using volatile (2010)

#60
post #8

volatile should only be used for accessing MMIO registers in device drivers; that's it.

There's definitely more uses. For example, shared memory between processes: you should mark it volatile. C++ atomics are no good here, because they are not guaranteed to be lock free or address free.

maybe use fences?
Post reply on HN