[deleted]
A tale of an impossible bug: big.LITTLE and caching
11–20 of 116 posts
Re: A tale of an impossible bug: big.LITTLE and caching
#12Earlier quoted context omitted.
Stupid question but does that work in a virtualised environment where your program can be live-migrated to another physical machine with a different CPU?
Nope. There's not really any alternative other than "Don't do that", or limit migration to machines that have a superset of the instructions on the original machine.
I presume that AWS or Azure wouldn't do that?
Re: A tale of an impossible bug: big.LITTLE and caching
#13It appears the that the caching code was added in this patch:
https://gcc.gnu.org/ml/gcc-patches/2012-09/msg00076.html
Prior to that, the call:
asm volatile ("mrs\t%0, ctr_el0":"=r" (cache_info));
Re: A tale of an impossible bug: big.LITTLE and caching
#14Earlier quoted context omitted.
Performance. get_current_cpu_cache_line_size would need to run some code to determine the cache line size, and that code takes longer to run than using a cached value. Along similar lines, if you have an optimized routine using specific CPU instructions, you don't want to call CPUID (or equivalent) on every call to find out if you have those instructions; you want to call it once and cache the answer. If it can retur…
Stupid question but does that work in a virtualised environment where your program can be live-migrated to another physical machine with a different CPU?
I believe virtual machines will typically alter the CPUID a bit:
https://tech.mendix.com/linux/2016/08/18/xen-cpuid-masking/
You'll choose a lowest common denominator of features sets.
Re: A tale of an impossible bug: big.LITTLE and caching
#15From the pseudo code, what is disadvantage that making get_current_cpu_cache_line_size() always get called?
It's slow, and also it doesn't fix the bug, as the code could get migrated immediately after calling it.
The only real fix is to do what they are doing -- always use the smallest line size of the system, regardless of which core you are running on.
Re: A tale of an impossible bug: big.LITTLE and caching
#16Huge props to the team for finding this out. What a nasty issue.
One question about the intro, which states that this is the first mass produced AMP architecture, but isn't the PlayStation 3's Cell CPU one?
Re: A tale of an impossible bug: big.LITTLE and caching
#17Earlier quoted context omitted.
Stupid question but does that work in a virtualised environment where your program can be live-migrated to another physical machine with a different CPU?
Nope. There's not really any alternative other than "Don't do that", or limit migration to machines that have a superset of the instructions on the original machine.
Usually you configure the VM to only report CPUID values corresponding to lowest-common-denominator features on everything you might want to migrate to. Then as long as the guest code plays nicely and looks at the CPUID feature flags to see what it can use, it'll migrate happily. QEMU has support for this, for instance.
Re: A tale of an impossible bug: big.LITTLE and caching
#18I don't understand that level of coding, what i do understand is the great way of debugging. Its all about deduction mr Watson.
Re: A tale of an impossible bug: big.LITTLE and caching
#19It appears the that the caching code was added in this patch: https://gcc.gnu.org/ml/gcc-patches/2012-09/msg00076.html Prior to that, the call: asm volatile ("mrs\t%0, ctr_el0":"=r" (cache_info)); was always made.
But the task can be rescheduled on a little core part-way through the execution...
The big cores should report the smaller cache line size always.
Re: A tale of an impossible bug: big.LITTLE and caching
#20Earlier quoted context omitted.
That's a question for the libgcc team. I seriously doubt it to be slow enough to matter.
it reads some coproc regs, which are instructions that cannot be reordered. they slow down everything on an OOO core. after that just some bitmasking (not slow)
For 64-bit ARMv8 (ie AArch64) system registers are in general reorderable; software must provide explicit synchronization (typically via barrier instructions) where it does not want the reordering, except for a few registers which have implicit synchronization. Since CTR_EL0 is entirely constant there's no inherent reason why it shouldn't be reorderable pretty freely, though it's an implementation detail how fast or otherwise it is in practice. (Benchmark if it matters to you!)
(This is all documented in the v8 ARM ARM section "Synchronization requirements for AArch64 System Registers".)