Live data from Hacker News

How to zero a buffer

daemonology.net

181–190 of 216 posts

Re: How to zero a buffer

#181

Does anyone have any advice on articles about C compiler optimizations in general (especially gcc)? I'm doing my first serious C work in ten years, and I keep wondering if I should fuss with things like this or let the compiler handle it all: foo->bar->baz[i].oof = foo->bar->baz[i].durb + meep; vs what *tmp = foo->bar->baz[i]; tmp->oof = tmp->durb + meep; EDIT: I'm not asking for a link to this: https://gcc.gnu.org/o…

Here's my advice. Generally speaking, the compiler is really smart. I would characterize the optimization you put as a third-grade optimization: GCC is in college (Clang too, for that matter). It's many steps ahead of that level.

However, if you're ever in doubt, I recommend compiling very short functions and viewing their output.

    typedef struct {
      int oof;
      int durb;
    } baz_t;

    typedef struct {
      baz_t *baz;
    } bar_t;

    typedef struct {
      bar_t *bar;
    } foo_t;

    void f(foo_t *foo, int i, int meep) {
      foo->bar->baz[i].oof = foo->bar->baz[i].durb + meep;
    }

    $ gcc -O2 -c -o test.o test.c
    $ objdump -d -r -M intel test.o

    test.o:     file format elf64-x86-64

    
    Disassembly of section .text:   
       
    0000000000000000 :           
       0:   48 8b 07                mov    rax,QWORD PTR [rdi]
       3:   48 63 f6                movsxd rsi,esi
       6:   48 8b 08                mov    rcx,QWORD PTR [rax]
       9:   48 8d 34 f1             lea    rsi,[rcx+rsi*8]
       d:   03 56 04                add    edx,DWORD PTR [rsi+0x4]
      10:   89 16                   mov    DWORD PTR [rsi],edx
      12:   c3
You can see here that it followed the chain of pointers only once.

The one thing to watch out for though is things that gcc isn't allowed to optimize because of C. For example, if a pointer escapes the function (to another function that the optimizer can't see), gcc cannot assume that the pointed-to memory remains unchanged, even if the called function takes a const pointer! Because the function could always cast away const. For example, this variant will have to follow the chain twice:

    int g(const foo_t *foo, int i);

    void f(foo_t *foo, int i) {
      int x = g(foo, foo->bar->baz[i].durb);
      foo->bar->baz[i].oof = x;
    }
Generally people always use at least -O2. The main difference between -O2 and -O3 is that -O3 is more aggressive with unrolling and other optimizations that increase code size, so sometimes -O2 is faster because of icache pressure. I generally use -O3 on my tightest loops and -O2 (or even -Os) on everything else.

Re: How to zero a buffer

#182

Interesting. This appears to solve a more general problem, which is: how to create a barrier against inter-procedural optimization and dead code elimination. I wonder if this trick could also be used to solve the double-checked locking problem. From the quintessential DCLP paper ( http://www.aristeia.com/Papers/DDJ_Jul_Aug_2004_revised.pdf ): Consider again the line that initializes pInstance: pInstance = new Singlet…

if the function must be called, then step 3 cannot possibly be performed before steps 1 and 2. So just to clarify, there's no way the compiler could do "1, 3, 2" instead of "1, 2, 3"? It seems a naive implementation of a compiler could store the pointer to the allocated memory in the pInstance variable before calling the constructor, rather than using a temporary location for the pointer (e.g. a register). Does C++11…

I should have been more specific. To use Colin's trick with this pattern, you would need to write a separate function (like InitializeSingleton()) that calls the constructor and returns the pointer. If InitializeSingleton() is impossible to inline/optimize, which is the goal of Colin's trick, then 1 and 2 must happen before 3, because 3 cannot happen until the function has been called and returns, and the function does steps 1 and 2.

Re: How to zero a buffer

#183
post #177

Does anyone have any advice on articles about C compiler optimizations in general (especially gcc)? I'm doing my first serious C work in ten years, and I keep wondering if I should fuss with things like this or let the compiler handle it all: foo->bar->baz[i].oof = foo->bar->baz[i].durb + meep; vs what *tmp = foo->bar->baz[i]; tmp->oof = tmp->durb + meep; EDIT: I'm not asking for a link to this: https://gcc.gnu.org/o…

foo->bar->baz[i].oof = foo->bar->baz[i].durb + meep; This is fine, no need to "optimize" anything. This kind of common subexpression elimination should be done by any modern compiler (for any language!) and the algorithm behind it is taught in university classes too. Most of the time it's safe to use -O3. If you're doing numerical code with floating points -ffast-math is also pretty safe if your code is correct (ie.…

>I usually use objdump -d objfile.o to look at assembly output

You can also compile to assembly with -S. I think it's clearer that way.

Re: How to zero a buffer

#184
post #177

Does anyone have any advice on articles about C compiler optimizations in general (especially gcc)? I'm doing my first serious C work in ten years, and I keep wondering if I should fuss with things like this or let the compiler handle it all: foo->bar->baz[i].oof = foo->bar->baz[i].durb + meep; vs what *tmp = foo->bar->baz[i]; tmp->oof = tmp->durb + meep; EDIT: I'm not asking for a link to this: https://gcc.gnu.org/o…

foo->bar->baz[i].oof = foo->bar->baz[i].durb + meep; This is fine, no need to "optimize" anything. This kind of common subexpression elimination should be done by any modern compiler (for any language!) and the algorithm behind it is taught in university classes too. Most of the time it's safe to use -O3. If you're doing numerical code with floating points -ffast-math is also pretty safe if your code is correct (ie.…

Really appreciate everyone's replies! A related question about my example: what if I want to assign the pointer dereference to `tmp` to improve readability (rather than avoid multiple traversals). Is there any reason not to use a tmp variable (presumably with a better name)?

Re: How to zero a buffer

#185
post #177

Earlier quoted context omitted.

foo->bar->baz[i].oof = foo->bar->baz[i].durb + meep; This is fine, no need to "optimize" anything. This kind of common subexpression elimination should be done by any modern compiler (for any language!) and the algorithm behind it is taught in university classes too. Most of the time it's safe to use -O3. If you're doing numerical code with floating points -ffast-math is also pretty safe if your code is correct (ie.…

Really appreciate everyone's replies! A related question about my example: what if I want to assign the pointer dereference to `tmp` to improve readability (rather than avoid multiple traversals). Is there any reason not to use a tmp variable (presumably with a better name)?

>what if I want to assign the pointer dereference to `tmp` to improve readability

I personally find code like that harder to follow. The first version is clearer than the second (and you forgot to take the address of foo->bar->baz[i]).

Re: How to zero a buffer

#186
post #177

Earlier quoted context omitted.

foo->bar->baz[i].oof = foo->bar->baz[i].durb + meep; This is fine, no need to "optimize" anything. This kind of common subexpression elimination should be done by any modern compiler (for any language!) and the algorithm behind it is taught in university classes too. Most of the time it's safe to use -O3. If you're doing numerical code with floating points -ffast-math is also pretty safe if your code is correct (ie.…

Really appreciate everyone's replies! A related question about my example: what if I want to assign the pointer dereference to `tmp` to improve readability (rather than avoid multiple traversals). Is there any reason not to use a tmp variable (presumably with a better name)?

> Is there any reason not to use a tmp variable (presumably with a better name)?

Nope -- shouldn't hurt at all.

It's interesting to me that LuaJIT recommends not using temp variables like this because they can hurt optimization for LuaJIT. That's obviously very different than C in almost every way, I just mention it because it was so surprising to me that there is a situation (in any optimized language) where a temp variable could hurt optimization.

Re: How to zero a buffer

#187
If your goal is just to "burn" the memory, why not write your own loop that copies some arbitrary piece of data that the compiler can't optimize out over the memory's contents? Do something like fill the buffer with its own pointer address.

Re: How to zero a buffer

#188

Earlier quoted context omitted.

Really appreciate everyone's replies! A related question about my example: what if I want to assign the pointer dereference to `tmp` to improve readability (rather than avoid multiple traversals). Is there any reason not to use a tmp variable (presumably with a better name)?

>what if I want to assign the pointer dereference to `tmp` to improve readability I personally find code like that harder to follow. The first version is clearer than the second (and you forgot to take the address of foo->bar->baz[i]).

> and you forgot to take the address of foo->bar->baz[i]

Ha, I was afraid of that. :-) Still re-learning when I need that with arrays and when not.

Re: How to zero a buffer

#189
post #94

Earlier quoted context omitted.

What about a data race? Theoretically, the function that memset_ptr points to could be changed between when it is checked and when it would be run.

If you have multiple threads accessing a shared (mutable) variable in your program, even a shared volatile variable, then you need to guard every access to that variable (which, in this case, includes every function call through memset_ptr) with proper thread synchronisation primitives. Marking a variable "volatile" is not enough to prevent data races in a multi-threaded environment. If you've put a semaphore, or mut…

I think you missed my point.

memset_ptr is a const (not changed by this program... theoretically) volatile (allowed to be changed by the system, theoretically) pointer to memset. In THIS PARTICULAR CASE, memset_ptr points to memset. The compiler however doesn't know that it won't change due to another processes, but we do. So the compiler shouldn't be able to optimize out the call directly to the function pointer because it introduces a possible race condition: the program reads that memset_ptr points to memset, then the pointer changes (due to some other process changing it), but the program still calls memset, and not memset_ptr. The optimization allows for a possible race condition to occur.

Re: How to zero a buffer

#190

When this still doesn't work: JIT compiled C. The compiler can check for memset and elide it. (Or hell, one can envision the hypothetical Antagonizer9000 compiler including a version of memset which peeks up the stack to see what it's clearing and stops short.)

Even if a JIT compiler can prove that all the code in your app doesn't change that function pointer, because the variable is volatile, the compiler must assume that you intend to read from actual metal every time you refer to it and it can not predict what the value will be. Even a JIT compiler is not allowed to optimize away that read, or else you'd never be able to write a driver.

Incorrect.

Because a JIT may have enough knowledge of the underlying system to know that the pointer is not pointing to a memory-mapped / DMA'd / etc area, and as such can be assumed to remain constant.

Post reply on HN