Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CPU Cache Essentials

meta-x86.blogspot.com

Post reply on HN
Print view View original

11–19 of 19 posts

Re: CPU Cache Essentials

#11

Post by gioele »

> use as much of cache line as possible.

> For example in the next code when else clause is happening, we are throwing complete cache line which was fetched by accessing the is_alive member

    struct Obj {
        bool is_alive;
        …
    };

    std::vector objs;

    for (auto o: objs) {
        if (o.is_alive)
            do_stuff(o);
        else {
            // just thrown a cache line
        }
    }
So, how do you fix that?

My first idea would be to remove the is_alive member from the objects and creating a is_alive array of boolean, one for each object. But this opens a whole new set of problems (how to map from objects to is_alive indexes, static vs dynamic set of object that can be alive, etc.).

Re: CPU Cache Essentials

#12

Post by Arelius »

gioele wrote: post #11

> use as much of cache line as possible. > For example in the next code when else clause is happening, we are throwing complete cache line which was fetched by accessing the is_alive member struct Obj { bool is_alive; … }; std::vector objs; for (auto o: objs) { if (o.is_alive) do_stuff(o); else { // just thrown a cache line } } So, how do you fix that? My first idea would be to remove the is_alive member from the obj…

Or even better, how about we just have an array of only alive objects.

Re: CPU Cache Essentials

#13

Post by Arelius »

phserr wrote: post #4

I see cache related optimizations being discussed most often in the context of C or C++. Being inexperienced in this area, I wonder how would does it affect garbage collected languages/systems. I understand that that are different approaches to GC, but can someone recommend some discussion to read on this topic? (cache optimization either in GC development itself or in development with GC languages)

The same principals apply, it just becomes more difficult to understand what your language may be doing for you in this regard.

Compound types in most languages will store their members contiguously, at least in cases where easy to do so. And arrays are generally contiguous in addition.

The problems often crop up in not being able to have compound value-types and are instead often included by reference.

Some languages, including C# IIRC, allow you to define specific types to be treated by value (structs in C#) which can often help. It may also be better to have a few arrays with primitive types in them, rather than a single array of compound types, for instance.

Re: CPU Cache Essentials

#14

Post by bainsfather »

astigsen wrote: post #2

Direct link to the original presentation by Scott Meyers: http://vimeo.com/97337258 (the article is just a summary)

I tried watching the talk, but he goes very slowly - could be compressed 3x easily.

The summary gives you the content of the talk much quicker.

Re: CPU Cache Essentials

#15

Post by voidlogic »

dbaupp wrote: post #10
voidlogic wrote: post #9

Earlier quoted context omitted.

Just to you are aware, you are presenting a false dichotomy. There are plenty of languages that are both garbage collected and give you quite a bit of control about memory layout (D, Go, Rust, etc). For example, in Go if I wanted have a slice of Foobar ([] Foobar) but wanted to have them allocated close to one another I could write: const neededFoobars = 100 backingSlice := make([]Foobar, neededFoobars) seqfoobarPtrs…

(FWIW, Rust is not garbage collected.)

Rust has optional garbage collection

Re: CPU Cache Essentials

#17

Post by ANTSANTS »

gioele wrote: post #11

> use as much of cache line as possible. > For example in the next code when else clause is happening, we are throwing complete cache line which was fetched by accessing the is_alive member struct Obj { bool is_alive; … }; std::vector objs; for (auto o: objs) { if (o.is_alive) do_stuff(o); else { // just thrown a cache line } } So, how do you fix that? My first idea would be to remove the is_alive member from the obj…

If you really want to optimize data cache usage, use a bitvector and (either through compiler intrinsics or inline assembly) your CPU's bit-scanning instructions. This can reduce the storage requirement to 1 bit per object and skip up to 32 or 64 not-alive objects in a single instruction.

https://news.ycombinator.com/item?id=6176091

Re: CPU Cache Essentials

#18

Post by devbug »

ANTSANTS wrote: post #17
gioele wrote: post #11

> use as much of cache line as possible. > For example in the next code when else clause is happening, we are throwing complete cache line which was fetched by accessing the is_alive member struct Obj { bool is_alive; … }; std::vector objs; for (auto o: objs) { if (o.is_alive) do_stuff(o); else { // just thrown a cache line } } So, how do you fix that? My first idea would be to remove the is_alive member from the obj…

If you really want to optimize data cache usage, use a bitvector and (either through compiler intrinsics or inline assembly) your CPU's bit-scanning instructions. This can reduce the storage requirement to 1 bit per object and skip up to 32 or 64 not-alive objects in a single instruction. https://news.ycombinator.com/item?id=6176091

As Arelius mentioned, it is more performant to just keep an array of alive objects as you're only touching memory that needs to be touched. That said, however, bitvictors/bitsets are incredibly useful if you have no easy or clean way of separating variations of objects.

An example would be "scene-graphs" in my game engine. Ostensibly they are stored as components (but my game engine doesn't have components) which are accessed by an unique identifier via an indirection table. Something like Bitsquid.[0]

Like Bitsquid, my "scene-graphs" can be linked to other "scene-graphs." This allows me to "glue" weapons to character's hands or armour to their torsos. This means I have "scene-graphs" that can be updated (nodes repositioned) in parallel (as they have no data dependencies) and others that cannot (linked "scene-graphs"). For performance reasons, I identify which "scene-graphs" can be updated in parallel, so I can kick them off to appropriate tasks which satisfy their data-dependencies. Thus enters the bitvector/bitset because copying my "scene-graphs" around and then adjusting other "scene-graphs" pointers (that point to the moved "scene-graph") when linking or unlinking results in a complete iteration (once if you batch) over all the "scene-graphs," or some form of indirection (some with memory fragmentation too!). Not to mention such approach complicates the code for handling "scene-graphs" a lot. Beyond that, I have to sort my "scene-graphs" (pointer list) by link depth when updating, so I don't update children before their parents, and the like. It's messy, but bitvectors/bitsets are a great solution.

I am experimenting with other methods to manage linked "scene-graphs" which fall under a split (linked/unlinked) array method, but I don't have anything yet, especially when I consider that I get a free iteration over all scene-graphs anyways (which hides a lot of the cost of the current method).

[0]: http://bitsquid.blogspot.ca/2011/09/managing-decoupling-part...

Edit: And I completely forgot to mention the performance concerns of having multiple cores touch the same memory: irregular stalls!

Re: CPU Cache Essentials

#19

Post by dbaupp »

voidlogic wrote: post #15
dbaupp wrote: post #10

Earlier quoted context omitted.

(FWIW, Rust is not garbage collected.)

Rust has optional garbage collection

It doesn't. There is a type called Gc that may be garbage collected in future, but at the moment it is just a bad reference counted pointer (the Rc type is better if you want reference counting). And it's not even guaranteed that a GC will ever be implemented, Rust provides other abstractions so GC would be rarely used even if it was implemented.
Post reply on HN