Ways to break your systems code using volatile (2010)
blog.regehr.org
Ways to break your systems code using volatile (2010)
1–10 of 77 posts
Re: Ways to break your systems code using volatile (2010)
#2Ironically volatile is just as bad in Java for different reasons. Frequently used for "lock free" synchronization, its usually actually worse than using locks because it can't be cached between cores. The variable is always loaded from main memory, which is usually much worse than holding a lock mutex in registers.
Re: Ways to break your systems code using volatile (2010)
#3You really should just use volatile for device drivers when accessing IO space with side-effects. Do not use volatile to build your own synchronization primitives.
Re: Ways to break your systems code using volatile (2010)
#4This should say 2010. I believe much of it is out of date, as C11 does have a memory model, and does provide both atomics and barriers. Many, if not most, uses of volatile should probably be replaced by atomics.
Re: Ways to break your systems code using volatile (2010)
#5Ironically volatile is just as bad in Java for different reasons. Frequently used for "lock free" synchronization, its usually actually worse than using locks because it can't be cached between cores. The variable is always loaded from main memory, which is usually much worse than holding a lock mutex in registers.
The standard pattern for working with atomics in java (volatiles are of limited use without atomic field updaters or varhandles) is to read it into a local variable, operate on that and only write it back to the volatile once you're done.
That has many benefits, among them the ability to store its value in registers.
Re: Ways to break your systems code using volatile (2010)
#6You really should just use volatile for device drivers when accessing IO space with side-effects. Do not use volatile to build your own synchronization primitives.
Don't forget about memory barriers. Otherwise your driver will fail on other CPU architectures.
Just because it works on x86, doesn't mean it works on ARM, MIPS, POWER or RISC-V. CPUs other than x86 can reorder stores with other stores and loads with other loads. It can cause the CPU to do the store that starts DMA before the stores that set up length and address are done!
Or just use C11 or C++11 memory model. Although those are still not available in too many cases, curse of having to use an ancient compiler...
Re: Ways to break your systems code using volatile (2010)
#7Edit: Looks like the slides had an inaccuracy (see replies). Huh, looks like I learned something today :)
I think a good way of summarizing volatile is this slide from my parallel architectures class [1]:
> Class exercise: describe everything that might occur during the
> execution of this statement
> volatile int x = 10
>
> 1. Write to memory
>
> Now describe everything that might occur during the execution of
> this statement
> int x = 10
>
> 1. Virtual address to physical address conversion (TLB lookup)
> 2. TLB miss
> 3. TLB update (might involve OS)
> 4. OS may need to swap in page to get the appropriate page
> table (load from disk to physical address)
> 5. Cache lookup (tag check)
> 6. Determine line not in cache (need to generate BusRdX)
> 7. Arbitrate for bus
> 8. Win bus, place address, command on bus
> 9. All caches perform snoop (e.g., invalidate their local
> copies of the relevant line)
> 10. Another cache or memory decides it must respond (let’s
> assume it’s memory)
> 11. Memory request sent to memory controller
> 12. Memory controller is itself a scheduler
> 13. Memory controller checks active row in DRAM row buffer.
> (May > need to activate new DRAM row. Let’s assume it does.)
> 14. DRAM reads values into row buffer
> 15. Memory arbitrates for data bus
> 16. Memory wins bus
> 17. Memory puts data on bus
> 18. Requesting cache grabs data, updates cache line and tags,
> moves line into exclusive state
> 19. Processor is notified data exists
> 20. Instruction proceeds
> * This list is certainly not complete, it’s just
> what I came up with off the top of my head.
I think a lot of the confusion stems from the illusion that a uniprocessor + in-order execution model implies to programmers who have never dealt with system-level code. I think in the future, performant software will require a bit more understanding of the underlying hardware on the part of your average software developer -- especially when you care about any sort of parallelism. It doesn't help that almost all common CS curriculum ignores parallelism until the 3rd year or more.
[1] http://www.cs.cmu.edu/~418/lectures/12_snoopimpl.pdf - the last 2 slides
Re: Ways to break your systems code using volatile (2010)
#8volatile should only be used for accessing MMIO registers in device drivers; that's it.
Re: Ways to break your systems code using volatile (2010)
#9You really should just use volatile for device drivers when accessing IO space with side-effects. Do not use volatile to build your own synchronization primitives.
[deleted]
Re: Ways to break your systems code using volatile (2010)
#10Edit: Looks like the slides had an inaccuracy (see replies). Huh, looks like I learned something today :) I think a good way of summarizing volatile is this slide from my parallel architectures class [1]: > Class exercise: describe everything that might occur during the > execution of this statement > volatile int x = 10 > > 1. Write to memory > > Now describe everything that might occur during the execution of > thi…
I don't understand these slides. The volatile keyword does not magically bypass the mechanism by which modern CPUs write to main memory. Am I missing something, or are they somehow meant to be ironic?