Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Common Systems Programming Optimizations and Tricks

paulcavallaro.com

Post reply on HN
Print view View original

21–30 of 98 posts

Re: Common Systems Programming Optimizations and Tricks

#21

Post by vardump »

ibrault wrote: post #18
vardump wrote: post #10

Very good article, facts looked correct and it had useful advice. I'd add, keep things local. Don't access memory (or cache) outside core (L1 & L2), NUMA region or processor socket boundary unnecessarily. Keep networking, GPU, etc. code in same NUMA region where the physical adapters are. Use memory like tape, stream through. CPU branch predictors love that kind of access pattern. Oh, and perhaps most importantly: us…

Could you elaborate on what it means to "use memory like tape"?

Sequential access patterns, forward or backward. Repeating predictable gaps are ok, but do remember minimum unit that can be read from memory is a cache line. So if you read one byte, you'll read 64 bytes on modern x86.

Re: Common Systems Programming Optimizations and Tricks

#22

Post by ajross »

ibrault wrote: post #18
vardump wrote: post #10

Very good article, facts looked correct and it had useful advice. I'd add, keep things local. Don't access memory (or cache) outside core (L1 & L2), NUMA region or processor socket boundary unnecessarily. Keep networking, GPU, etc. code in same NUMA region where the physical adapters are. Use memory like tape, stream through. CPU branch predictors love that kind of access pattern. Oh, and perhaps most importantly: us…

Could you elaborate on what it means to "use memory like tape"?

DRAM latency for a random access read can get into the low hundreds of cycles on modern multi-socket devices. But the streaming bandwidth remains very high. Cache systems will routinely prefetch the next block ahead of an access if they detect that memory is being used sequentially, eliminating a huge chunk of that pipeline stall.

Re: Common Systems Programming Optimizations and Tricks

#23

Post by SlySherZ »

For everyone that enjoyed this, there's an entire free online MIT course called Performance Engineering of Software Systems[1] where you'll learn plenty more tricks and common pitfalls like these. You'll also learn how to use tools to debug the low level performance of your programs: looking at cache misses, cpu utilization, time spent per assembly operation and so on. It's pretty cool :)

[1] https://ocw.mit.edu/courses/electrical-engineering-and-compu...

Re: Common Systems Programming Optimizations and Tricks

#24

Post by ajross »

notacoward wrote: post #11
jcelerier wrote: post #7

Earlier quoted context omitted.

> As someone who's worked on old Macs and has also done lots of 32 -> 64-bit porting, this is the sort of trick that works wonderfully...until it doesn't. And then you've got a nightmare on your hands. That's why you hide the trick behind a zero-cost abstraction which checks at compile-time if the platform supports this

One of my favorite system programming tricks is to never believe that a "zero cost abstraction" lives up to the name.

Yeah, but an even more important system programming trick is to measure what you're doing every time. Performance optimization at this level is never about just trusting tools. If you aren't willing to be reading generated machine code and checking perf counters, you aren't really going to get much benefit.

And if you're willing to check your optimizations by reading disassembly, tricks like stuffing tag bits into the bottom of aligned pointer values is pretty routine.

Re: Common Systems Programming Optimizations and Tricks

#26

Post by saagarjha »

holy_city wrote: post #25

The false-sharing macro in the example expands to __attribute__((alligned(/* etc / )) or __declspec(align(/ etc*/). Is there a reason these are preferred over the alignas specifier introduced in C++ 11?

I believe there's a note that recommends the use of alignas when available: https://github.com/abseil/abseil-cpp/blob/fa00c321073c7ea40a...

Re: Common Systems Programming Optimizations and Tricks

#27

Post by notacoward »

jjuhl wrote: post #15
notacoward wrote: post #11

Earlier quoted context omitted.

One of my favorite system programming tricks is to never believe that a "zero cost abstraction" lives up to the name.

Modern optimizing C++ compilers (especially with Link Time Optimization enabled) are pretty amazing and can very often actually achieve that abstraction collapsing.. But, of course, always measure.

While it's true that modern compilers are wondrous things, checking whether they're clever enough to optimize away a particular construct - and to do so correctly, and to continue doing so in the next release - still takes time. If the same optimization can be done at a higher level, such that it will apply for any correct (but not necessarily clever) compiler, that's preferable. In my experience that's practically all the time. The best compiler optimizations IMO are the ones that can't be easily done at the source level.

Re: Common Systems Programming Optimizations and Tricks

#28

Post by kabdib »

"Repurposing Top Bits" - don't do that. Honest.

The IBM 360 shipped with 32-bit addresses but only 24 bits decoded. "Hey, there's a whole byte up top that nobody's using today, let's put some stuff there!" When they wanted the address space IBM found themselves architecturally hamstrung, and the cost to dig out was significant.

The 128K Macintosh used a 68000; it had 32-bit addresses but only 24 bits were decoded. "Hey, there's a whole byte up top that nobody's using today, let's put some stuff there!" When Apple needed the address space they found themselves hamstrung by pieces of MacOS that did use those bits, and many applications that did, too. The cost to dig out was significant.

It is practically guaranteed that "Hey, there's 16 whole bits up there that nobody's using today" will wind up the same, because this industry just never learns.

You can do things with lower bits and usually get away with it; many systems put GC tags and type bits down there. But those upper address bits do not belong to you.

Re: Common Systems Programming Optimizations and Tricks

#29

Post by Symmetry »

In most cases your compiler should do the clever work of turning your division or modulo operation into easier to do bit banging... but only if you're operating on a reasonable constant. Powers of two are best but you can do other constant divisions by stringing together 1-latency operations in ways that are still far faster than division.

Re: Common Systems Programming Optimizations and Tricks

#30

Post by astrobe_ »

groby_b wrote: post #12
legulere wrote: post #3

Instead of repurposing top bits you can also repurpose the Bits beyond alignment. E.g 32 bit integers are aligned to 4 bytes, so you can use the lower two bits of pointers to them instead.

That does not hold for Intel x86 architecture chips, which are perfectly happy to have unaligned integers. As for struct members, the alignment is (of course) "implementation defined", which is the fancy way of throwing up your hands and saying "whatever". (Since C++11 we actually have alignas(), which at least gives manual control)

One can construct an aligned pointer on a chunk of allocated memory by asking for the needed quantity plus the size of the alignment and the "nudge" the pointer you get to the next word boundary (and you can do something with the "wasted" byte before it, like storing the size of the object).

But that sort of trick is only worth it if you are writing a compiler and its runtime, an interpreter, a memory allocator (in particular GCs) or at the very last some sort of high performance library (you would return your "featured pointer" as an opaque type to the user).

Post reply on HN