Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Myths Programmers Believe about CPU Caches (2018)

software.rajivprab.com

Post reply on HN
Print view View original

71–80 of 143 posts

Re: Myths Programmers Believe about CPU Caches (2018)

#71

Post by brabel »

pajko wrote: post #48
garfieldandthe wrote: post #45

The central myth is that the average programmer should care. The typical programmer should treat CPU caches as what they are designed to be: mostly transparent. You work in a high level language and leave the tricky details to a library and your compiler. It's only a small minority that should really worry about these things. In my daily work, I see more often premature microoptimizations (in part using the myths fro…

This is partly wrong. For example, it's important how the fields are organized in a structure, and only the developer knows what belongs together and only they know the access patterns across threads (or at least should know). This is far from being a micro-optimization.

If I do it wrong, are we talking about my application (say, some web app) becoming noticeably slower? Or we would need to be running several million operations per second before anything a user could notice the difference?

I suspect only very high end games, image processing apps and that kind of thing would ever need to care about the order of fields in a struct... perhaps the author of my web server as well, but even that only if I have a ridiculously high load on my server (which I don't, almost no one does).

So, no I don't think I should know that until I develop one of those highly niche applications myself.

Re: Myths Programmers Believe about CPU Caches (2018)

#75

Post by harry8 »

ggm wrote: post #5

My favourite myth is that I still believe it's possible to write code which operates totally out of L1 and L2 cache. Not just tight ASM on bare metal: C code or similar, compiled down, to run under a modern UNIX/POSIX os on a multi-core host. I have never explored HOW this would work, or WHAT I would do to achieve it, but I believe it, implicitly. For it to be true I would have to understand the implications of every…

I guess that means my favourite myth that I've done it. Get the os out of the way, cpu pinning, user level network stack, no system calls after setup, lay out all your memory carefully in a cache line aware fashion noting the virtual addresses & bits 6-10. io via shared memory to another core on the same package.

I convinced myself looking at latency benchmarks as I went about optimising it that it was all in L1.

Re: Myths Programmers Believe about CPU Caches (2018)

#76

Post by notacoward »

jcranmer wrote: post #26
notacoward wrote: post #23

Earlier quoted context omitted.

Oh, you suppose the authors spent their time writing that for no reason ? You know better than them? Your explanation is so incomplete I hardly know where to begin. Maybe with the fact that you're talking about local ordering as it may or may not be perturbed by a compiler, while the document is talking about the observed order elsewhere (other processors or main memory). That observed order can vary depending on the…

> Oh, you suppose the authors spent their time writing that for no reason? Your explanation is so incomplete I hardly know where to begin. No. I am suggesting that you don't need the full rigor of understanding the memory barrier semantics to be able to effectively write multithreaded code correctly. Completeness was never a goal of my explanation; sufficiency was. Furthermore, my focus is on a software memory model,…

> dominant memory model in use by, well, everybody.

"...by, well, x86"

Fixed that for you.

Re: Myths Programmers Believe about CPU Caches (2018)

#77

Post by SomeHacker44 »

ggm wrote: post #5

My favourite myth is that I still believe it's possible to write code which operates totally out of L1 and L2 cache. Not just tight ASM on bare metal: C code or similar, compiled down, to run under a modern UNIX/POSIX os on a multi-core host. I have never explored HOW this would work, or WHAT I would do to achieve it, but I believe it, implicitly. For it to be true I would have to understand the implications of every…

ISTR that is how a modern CPU will boot, before the DRAM interfaces have been calibrated and brought up. The initial startup code uses cache as RAM.

Re: Myths Programmers Believe about CPU Caches (2018)

#78

Post by stephencanon »

Tuna-Fish wrote: post #9
edrxty wrote: post #6

Earlier quoted context omitted.

Modern CPUs are pushing 1MB L1 and 8MB L2 or more. You can fit a dozen FreeRTOS instances in that space. It would be pretty cool to see someone build a system that used a high end CPU but didn't have any installed ram, though I'm not sure if the CPU microcode would be ok with that.

Those numbers are for all cores. No modern CPU has anywhere near that of L1 and L2 per core.

Apple’s p-cores have 192kB L1 (data, another 128k i$) and 4MB L2 per core. The L2 number is at least “close” to what GP said, I guess.

Re: Myths Programmers Believe about CPU Caches (2018)

#79

Post by jcranmer »

notacoward wrote: post #76
jcranmer wrote: post #26

Earlier quoted context omitted.

> Oh, you suppose the authors spent their time writing that for no reason? Your explanation is so incomplete I hardly know where to begin. No. I am suggesting that you don't need the full rigor of understanding the memory barrier semantics to be able to effectively write multithreaded code correctly. Completeness was never a goal of my explanation; sufficiency was. Furthermore, my focus is on a software memory model,…

> dominant memory model in use by, well, everybody. "...by, well, x86" Fixed that for you.

No, you didn't. It's the C++ memory model, which is borrowed by C, then every compiler IR targeting something in the C/C++ space, and then every other language that decided to support language-level atomics.

Your mistake is thinking solely in terms of the hardware memory model. Indeed, if you care only about x86 (and ignore the potential for compiler optimizations), most of the variety provided by the C/C++ memory model is unnecessary, as every operation on the x86 (well, except for the exceptions) are inherently release/acquire operations.

Please do read up on the C/C++ memory model added in C11 and C++11. It may serve you to understand this space better.

Re: Myths Programmers Believe about CPU Caches (2018)

#80

Post by jcranmer »

gpderetta wrote: post #47
jcranmer wrote: post #21

Earlier quoted context omitted.

You don't need that full documentation. Really, I could simplify what most programmers would need to understand about memory ordering down to this text: There are a few models of cross-thread memory ordering that you can choose between. If you have never been exposed to this field before, the naïve model of memory ordering you probably think is going on is sequential consistency. This is not implemented in hardware b…

> You get to pretend everything is sequentially consistent if you write proper synchronization. The easiest way to satisfy proper synchronization is an acquire-release model But acquire/release give, generally,a partial ordering, not a total order like sequential consistency. That's often enough of course.

The data-race-free theorem states that, in the absence of data races, acquire/release is indistinguishable from sequential consistency. Define data races to be UB, as C/C++ do, and you get to the state that acquire/release lets you pretend everything (except atomic operations themselves) is sequentially consistent.
Post reply on HN