Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A tale of an impossible bug: big.LITTLE and caching

mono-project.com

Post reply on HN
Print view View original

61–70 of 116 posts

Re: A tale of an impossible bug: big.LITTLE and caching

#61

Post by comex »

wolfgke wrote: post #54
tveita wrote: post #50

Earlier quoted context omitted.

That's too specific a feature to expose to developers - most people wouldn't even know about the feature, and the ones that did might needlessly enable it just to be conservative. The intention of the big.LITTLE architecture is to let processes be migrated seamlessly between the small and big core and let the unused core be turned off to save power. The kernel and the hardware should work together to make the core sw…

> The intention of the big.LITTLE architecture is to let processes be migrated seamlessly between the small and big core and let the unused core be turned off to save power. On the other hand flushing specific cache lines is a rather special feature. If the code uses such obscure low-level features (much more than the "typical" application) such as the invariant that the size of a cache-line stays constant over the e…

Anything that uses a JIT necessarily uses this "obscure low-level feature", including any program implemented in one of many programming languages. Forcing such programs onto the heavy processor makes no sense; there just needs to be a way to properly clear the cache.

Re: A tale of an impossible bug: big.LITTLE and caching

#62

Post by wyldfire »

pm215 wrote: post #26
pm215 wrote: post #10

Properly configured big.LITTLE clusters should be set up so that all CPUs report the same cache line size (which might be smaller than the true cache line size for some of the CPUs), to avoid exactly this kind of problem. The libgcc code assumes the hardware is correctly put together. There is a Linux kernel patchset currently going through review which provides a workaround for this kind of erratum by trapping the C…

Also, if I'm reading the proposed fix in the mono pull request correctly, it doesn't deal with the problem entirely because there's a race condition where the code might start execution on the core with the larger cache line size, and then get context-switched to the core with the smaller cache line size midway through executing its cache-maintenance loop. The chances of things going wrong are much smaller, but they'…

It should converge to the right value eventually but does seem like there's definitely a chance for it to be wrong one or more times before running on the little cores.

Re: A tale of an impossible bug: big.LITTLE and caching

#64

Post by brandmeyer »

pm215 wrote: post #29
ChuckMcM wrote: post #27

wow, just wow. That is a really awesome bug (and like the authors I have issues with trying to sleep when that sort of puzzle is sitting there :-) Still a bit hazy on why they manually flush the cache for a given block of memory (presumably for protecting disclosure?) but I'm also a bit curious how it works if you get the sequence big fetches a cache line, switches to little which fetches a line (half as long and cha…

Handling of the case where big and little both want the same thing in their cache should be dealt with by the usual cache-coherency traffic between the CPUs that ensures they don't disagree about what's in their L1 caches (very handwaved because I don't know the details). The reason for the manual cache operations is because they're generating JITted code -- on ARM to ensure that what you execute is the same thing yo…

I don't see why they have to do this in userspace at all. If they did:

* allocate read/write buffer

* JIT instructions into it

* change mapping to read/execute

* run the JITted code

Then the kernel manages flushing the data caches on the mapping change, and Mono gets to wrap a Somebody Else's Problem field around it. It sounds like they are instead:

* allocate read/write/execute buffer

* JIT instructions into it

* manually flush relevant data caches (with an assumption that the cache line size is constant)

* run the JITted code

Re: A tale of an impossible bug: big.LITTLE and caching

#65

Post by filereaper »

pm215 wrote: post #29
ChuckMcM wrote: post #27

wow, just wow. That is a really awesome bug (and like the authors I have issues with trying to sleep when that sort of puzzle is sitting there :-) Still a bit hazy on why they manually flush the cache for a given block of memory (presumably for protecting disclosure?) but I'm also a bit curious how it works if you get the sequence big fetches a cache line, switches to little which fetches a line (half as long and cha…

Handling of the case where big and little both want the same thing in their cache should be dealt with by the usual cache-coherency traffic between the CPUs that ensures they don't disagree about what's in their L1 caches (very handwaved because I don't know the details). The reason for the manual cache operations is because they're generating JITted code -- on ARM to ensure that what you execute is the same thing yo…

+1

Not just that, the ISA usually requires that a cache invalidation instruction be issued regardless of whether the chip's coherency will automatically detect and invalidate it.

In cases such as this post, it is perfectly valid for the silicon engineers to say that its the software's fault for not adhering to the ISA.

Re: A tale of an impossible bug: big.LITTLE and caching

#66

Post by maximilianburke »

Animats wrote: post #58

"first mass produced AMP architecture" Nope. Remember the Cell? The processor in the Playstation 3? One main CPU with 8 little CPUs and no shared memory, just channels. The Playstation 4 isn't a AMP machine because programming the Cell was so hard.

> The Playstation 4 isn't a AMP machine because programming the Cell was so hard.

The PS4 could be an AMP design if you consider the (closely coupled) GPU a processor.

It doesn't require the same gymnastics as the PS3 though because both the main processor and GPU are more capable. The SPUs were required to perform computation that the anemic PPU could not do as well as fill in where the pre-unified shader model GPU was unable to keep the pace.

Memory was shared on the PS3 but, from the SPUs, required explicit put and fetch operations.

Re: A tale of an impossible bug: big.LITTLE and caching

#67

Post by filereaper »

brandmeyer wrote: post #64
pm215 wrote: post #29

Earlier quoted context omitted.

Handling of the case where big and little both want the same thing in their cache should be dealt with by the usual cache-coherency traffic between the CPUs that ensures they don't disagree about what's in their L1 caches (very handwaved because I don't know the details). The reason for the manual cache operations is because they're generating JITted code -- on ARM to ensure that what you execute is the same thing yo…

I don't see why they have to do this in userspace at all. If they did: * allocate read/write buffer * JIT instructions into it * change mapping to read/execute * run the JITted code Then the kernel manages flushing the data caches on the mapping change, and Mono gets to wrap a Somebody Else's Problem field around it. It sounds like they are instead: * allocate read/write/execute buffer * JIT instructions into it * ma…

JITs need to provide read/write/execute access to JIT'ed bodies, particularly for things like Polymorphic Inline Cache's (PICs : https://en.wikipedia.org/wiki/Inline_caching#Polymorphic_inl...)

In which case a manual flush of the icache is needed, hence this problem.

Re: A tale of an impossible bug: big.LITTLE and caching

#68

Post by rodrigokumpera »

brandmeyer wrote: post #64
pm215 wrote: post #29

Earlier quoted context omitted.

Handling of the case where big and little both want the same thing in their cache should be dealt with by the usual cache-coherency traffic between the CPUs that ensures they don't disagree about what's in their L1 caches (very handwaved because I don't know the details). The reason for the manual cache operations is because they're generating JITted code -- on ARM to ensure that what you execute is the same thing yo…

I don't see why they have to do this in userspace at all. If they did: * allocate read/write buffer * JIT instructions into it * change mapping to read/execute * run the JITted code Then the kernel manages flushing the data caches on the mapping change, and Mono gets to wrap a Somebody Else's Problem field around it. It sounds like they are instead: * allocate read/write/execute buffer * JIT instructions into it * ma…

That approach is harder to use in practice that in sounds. It's not like people have not tried it.

The OS only let you alloc in large granules, like 4k or 16k, and the vast majority of the methods are significantly smaller than that, meaning a JIT must colocate multiple methods in the same allocation block or waste a significant amount of memory.

We could get around that by remapping memory between read/write to read/execute and have the OS solve the problem for us. Except for a couple of small details, modifying a memory mapping is very expensive and we're, well, in the performance business, and that mono is multi-threaded so one thread might be executing code from the exact page we just made non-executable.

This approach, IIRC, was tried by Firefox as it has some security advantages, but discarded due to the measurable performance impact - and they don't have the second problem as JS is single threaded.

Full Disclosure: I'm part of the Mono team.

Re: A tale of an impossible bug: big.LITTLE and caching

#69

Post by wolfgke »

lambda wrote: post #60
wolfgke wrote: post #55

Earlier quoted context omitted.

> But then you would never be able to take advantage of the power/performance tradeoff that big.LITTLE gives you Rather: Only applications that don't depend on the invariant that the size of a cache line will stay constant over the execution time will take advantage of this. I can imagine ways how one could increase the advantage, but I don't want to go too much into technical details here.

Why would programs want to "depend on the invariant that the size of a cache line will stay constant"? That's a minor detail that most applications don't care about. In this case, it probably mattered because when JITing code, you need to explicitly flush instruction cache after changing code. But that's not anything the application cares about; that's something the runtime cares about, and the runtime is going to wa…

> Also, from a user perspective, you don't want applications to be able to demand only the big, powerful cores.

This should rather be a setting in the rights system that a user can refuse the right to an application to be able to demand big, powerful cores. I can nevertheless imagine quite well scenarios where enabling a (specific) application to do so can be quite useful.

Re: A tale of an impossible bug: big.LITTLE and caching

#70

Post by wolfgke »

comex wrote: post #61
wolfgke wrote: post #54

Earlier quoted context omitted.

> The intention of the big.LITTLE architecture is to let processes be migrated seamlessly between the small and big core and let the unused core be turned off to save power. On the other hand flushing specific cache lines is a rather special feature. If the code uses such obscure low-level features (much more than the "typical" application) such as the invariant that the size of a cache-line stays constant over the e…

Anything that uses a JIT necessarily uses this "obscure low-level feature", including any program implemented in one of many programming languages. Forcing such programs onto the heavy processor makes no sense; there just needs to be a way to properly clear the cache.

On https://news.ycombinator.com/item?id=12483698 I wrote a better idea how this might be implemented without causing this problem. Nevertheless even if we use my first, worse interface this should not be a problem: Spawn a thread that does the lowlevel stuff, sync it and after that run the JITted code.
Post reply on HN