Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A tale of an impossible bug: big.LITTLE and caching

mono-project.com

Post reply on HN
Print view View original

1–10 of 116 posts

Re: A tale of an impossible bug: big.LITTLE and caching

#3

Post by JoshTriplett »

xxie24 wrote: post #2

From the pseudo code, what is disadvantage that making get_current_cpu_cache_line_size() always get called?

Performance. get_current_cpu_cache_line_size would need to run some code to determine the cache line size, and that code takes longer to run than using a cached value.

Along similar lines, if you have an optimized routine using specific CPU instructions, you don't want to call CPUID (or equivalent) on every call to find out if you have those instructions; you want to call it once and cache the answer. If it can return different answers on different CPUs in the same system, you need to use a different mechanism instead, such as asking the OS for the least common denominator features of available CPUs, or notifying the OS that you need to run on one of the more capable CPUs that has the feature you need.

Re: A tale of an impossible bug: big.LITTLE and caching

#5

Post by dmitrygr »

rodrigokumpera wrote: post #4
xxie24 wrote: post #2

From the pseudo code, what is disadvantage that making get_current_cpu_cache_line_size() always get called?

That's a question for the libgcc team. I seriously doubt it to be slow enough to matter.

it reads some coproc regs, which are instructions that cannot be reordered. they slow down everything on an OOO core. after that just some bitmasking (not slow)

Re: A tale of an impossible bug: big.LITTLE and caching

#6

Post by _ihaque »

xxie24 wrote: post #2

From the pseudo code, what is disadvantage that making get_current_cpu_cache_line_size() always get called?

That would create a race condition addressed at the bottom of the article: the process can get switched onto another CPU between the invocation of get_current_cpu_cache_line_size() and the invalidation.

  An astute reader might realize that computing the cache line on every
  invocation is not enough for user space code: It can happen that a process
  gets scheduled on a different CPU while executing the __clear_cache
  function with a certain cache line size, where it might not be valid
  anymore.

Re: A tale of an impossible bug: big.LITTLE and caching

#7

Post by cm2187 »

JoshTriplett wrote: post #3
xxie24 wrote: post #2

From the pseudo code, what is disadvantage that making get_current_cpu_cache_line_size() always get called?

Performance. get_current_cpu_cache_line_size would need to run some code to determine the cache line size, and that code takes longer to run than using a cached value. Along similar lines, if you have an optimized routine using specific CPU instructions, you don't want to call CPUID (or equivalent) on every call to find out if you have those instructions; you want to call it once and cache the answer. If it can retur…

Stupid question but does that work in a virtualised environment where your program can be live-migrated to another physical machine with a different CPU?

Re: A tale of an impossible bug: big.LITTLE and caching

#8

Post by anarazel »

Different cacheline sizes for the different cores seems like an absurdly bad idea. One because it opens one up to bugs like these, but also because it makes optimization a lot harder. I have a hard time believing the savings due to a larger line size are worth it.

Re: A tale of an impossible bug: big.LITTLE and caching

#9

Post by mjg59 »

cm2187 wrote: post #7
JoshTriplett wrote: post #3

Earlier quoted context omitted.

Performance. get_current_cpu_cache_line_size would need to run some code to determine the cache line size, and that code takes longer to run than using a cached value. Along similar lines, if you have an optimized routine using specific CPU instructions, you don't want to call CPUID (or equivalent) on every call to find out if you have those instructions; you want to call it once and cache the answer. If it can retur…

Stupid question but does that work in a virtualised environment where your program can be live-migrated to another physical machine with a different CPU?

Nope. There's not really any alternative other than "Don't do that", or limit migration to machines that have a superset of the instructions on the original machine.

Re: A tale of an impossible bug: big.LITTLE and caching

#10

Post by pm215 »

Properly configured big.LITTLE clusters should be set up so that all CPUs report the same cache line size (which might be smaller than the true cache line size for some of the CPUs), to avoid exactly this kind of problem. The libgcc code assumes the hardware is correctly put together.

There is a Linux kernel patchset currently going through review which provides a workaround for this kind of erratum by trapping the CTR_EL0 accesses to the kernel so they can be emulated with the safe correct value: http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg... and it seems to me that that's really the right way to deal with this.

Post reply on HN