Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to zero a buffer

daemonology.net

Post reply on HN
Print view View original

61–70 of 216 posts

Re: How to zero a buffer

#61

Post by davidtgoldblatt »

tedunangst wrote: post #53

When this still doesn't work: JIT compiled C. The compiler can check for memset and elide it. (Or hell, one can envision the hypothetical Antagonizer9000 compiler including a version of memset which peeks up the stack to see what it's clearing and stops short.)

To clarify: the strategy in the post doesn't actually work (or at least, is not guaranteed to work in every conforming implementation): the "volatile" only applies to the read of the function pointer, not to the execution of the function in question.

You don't even need to assume some sort of crazy evil compiler to have to worry about this - speculative inlining of function pointers guarded by a safety check is something that FDO builds will actually do.

Re: How to zero a buffer

#62

Post by jfindley »

cperciva wrote: post #43
pedrocr wrote: post #40

Earlier quoted context omitted.

Thanks. That fits my general idea of FreeBSD. I've mostly wondered about how they technically compare though. Facebook seems to think Linux is generally faster for example[1]. I'd love to follow this more closely but I haven't found a LWN equivalent for the BSDs. [1] https://lwn.net/Articles/608954/

I think the best answer there is "it depends". Facebook has an open job posting which states that "Our goal over the next few years is for the Linux kernel network stack to rival or exceed that of FreeBSD" [1], so clearly there's at least one place in Facebook where Linux does not provide the best performance... [1] https://www.facebook.com/careers/department?req=a0IA000000Cz...

I wonder when that was posted. Linux networking has improved massively since the 2.6 days, and continues to get better.

It's still not FreeBSD, but the difference is pretty small these days.

Re: How to zero a buffer

#63

Post by pwg »

DSMan195276 wrote: post #18
drivingmenuts wrote: post #13

Why would the compiler be allowed optimize away a call to a perfectly valid function? This seems like it's allowing to compiler to make judgement calls on whether or not your code is worthy of being executed.

It's because memset is a standard function and has a defined standard way of acting, with the most important part being that it doesn't produce any side-effects. It's also worth noting that accessing memory that's no longer in the current scope is undefined-behavior, so the compiler can assume it doesn't happen. Thus the memset has absolutely zero effect on the actual program and isn't necessary. In general this isn'…

What if one made some trivial use of the block of memory after having performed the memset, say something like this:

   void
   dosomethingsensitive(void)
   {
        uint8_t key[32];
        ...
        /* Zero sensitive information. */
        memset((volatile void *)key, 0, sizeof(key));
        key[0] = key[1] + 1;
   }
Would that thwart the optimizer, or would it also see through that usage and eliminate it as well?

Re: How to zero a buffer

#64

Post by BrandonM »

davidtgoldblatt wrote: post #61
tedunangst wrote: post #53

When this still doesn't work: JIT compiled C. The compiler can check for memset and elide it. (Or hell, one can envision the hypothetical Antagonizer9000 compiler including a version of memset which peeks up the stack to see what it's clearing and stops short.)

To clarify: the strategy in the post doesn't actually work (or at least, is not guaranteed to work in every conforming implementation): the "volatile" only applies to the read of the function pointer, not to the execution of the function in question. You don't even need to assume some sort of crazy evil compiler to have to worry about this - speculative inlining of function pointers guarded by a safety check is somet…

The first comment there (by Anonymous) claims that the final technique can also be optimized:

    (memset_ptr)(p, 0, len);
> can be replaced by:

    if (memset_ptr == memset) {
        memset(p, 0, len);
    } else {
        memset_ptr(p, 0, len);
    }
> Which in turn can be optimized using the other tricks noticed above into:

    if (memset_ptr != memset) {
        memset_ptr(p, 0, len);
    }
I'm no expert, but this seems like a believable defeat of the technique in the post.

Re: How to zero a buffer

#65

Post by bunderbunder »

schoen wrote: post #55
cperciva wrote: post #49

Earlier quoted context omitted.

The term "observable behaviour" is defined in the standard: Essentially, I/O to files and interactive devices, plus accesses to volatile objects.

Perhaps in retrospect this was an inappropriate choice of definition, at least for cryptographic operations.

I'm gonna go ahead and say it: Perhaps in retrospect C is an inappropriate choice of language for these kinds of applications.

This "Performance at all costs, including safety and predictability" thing may be appropriate in video games, but for security-critical applications that philosophy is downright negligent.

Re: How to zero a buffer

#66

Post by tedunangst »

bunderbunder wrote: post #65
schoen wrote: post #55

Earlier quoted context omitted.

Perhaps in retrospect this was an inappropriate choice of definition, at least for cryptographic operations.

I'm gonna go ahead and say it: Perhaps in retrospect C is an inappropriate choice of language for these kinds of applications. This "Performance at all costs, including safety and predictability" thing may be appropriate in video games, but for security-critical applications that philosophy is downright negligent.

I'm not aware of any language that would be better. Most languages don't even let you touch memory to try to zero it.

Re: How to zero a buffer

#67

Post by cperciva »

pwg wrote: post #63
DSMan195276 wrote: post #18

Earlier quoted context omitted.

It's because memset is a standard function and has a defined standard way of acting, with the most important part being that it doesn't produce any side-effects. It's also worth noting that accessing memory that's no longer in the current scope is undefined-behavior, so the compiler can assume it doesn't happen. Thus the memset has absolutely zero effect on the actual program and isn't necessary. In general this isn'…

What if one made some trivial use of the block of memory after having performed the memset, say something like this: void dosomethingsensitive(void) { uint8_t key[32]; ... /* Zero sensitive information. */ memset((volatile void *)key, 0, sizeof(key)); key[0] = key[1] + 1; } Would that thwart the optimizer, or would it also see through that usage and eliminate it as well?

That just creates another dead store which will get eliminated.

Re: How to zero a buffer

#68

Post by e12e »

It is a little mind boggling that support for proper handling of this didn't arrive until c11. For a symmetric cipher without a demanding setup/init phase - would it make sense to just do a few rounds on a buffer using the zeroed key? Obviously quite a few more cycles, but should at least be a predictable (constant) overhead?

Re: How to zero a buffer

#69

Post by alexforster »

defen wrote: post #23

Nice teaser at the end there. Does it have something to do with the fact that the OS may have paged the memory containing the sensitive data to disk?

I'm hoping it'll be an allusion to Rust and an article about how none of this nonsense is necessary, but I don't think so based on his past writing.

Re: How to zero a buffer

#70

Post by fafner »

denim_chicken wrote: post #46

In GNU C one can add the statement asm ("" : : "m" (&key)); just before or after the memset, effectively telling the compiler that the address of "key" escapes the scope of the function.

GCC also has an `optimize' function attribute which might make sense to use here. This can set optimizations for the function to -O0. But I haven't tried it.
Post reply on HN