Live data from Hacker News

How to zero a buffer

daemonology.net

101–110 of 216 posts

Re: How to zero a buffer

#101
post #85

The article the completely obvious: /* implemented in another translation unit */ void zero_for_sure(void *data, size_t size); void func(void) { char securedata[42]; /* ... */ zero_for_sure(securedata, sizeof securedata); } The key here is that our zero_for_sure is an external function in a separately translated file. In the absence of a stunningly advanced global optimization that peeks into other previously compile…

>In the absence of a stunningly advanced global optimization that peeks into other previously compiled units You'll be surprised, but this stuff exists since late last century, known as Link-Time Code Generation (LTCG): http://msdn.microsoft.com/en-us/magazine/cc301698.aspx

I've just been digging through the C99 standard (official) and the N1570 (final C11 draft) and have come to the conclusion that these optimizations break the language. This is probably not something you should be compiling your OpenSSL shared library or SSH with.

A C program consists of translation units which may be preserved in translation form. That happens in translation phases 1 through 7. Multiple translation units may be linked, which is translation phase 8. Phase 8 only consists of resolving references; the last semantic analysis takes place in phase 7.

An example under 5.1.2.3 gives the range of adherence between actual semantics and abstract semantics. Though it is just an example, and not normative, it is very clear from the wording that the locus of valid optimizations is the translation unit.

Selected citations:

5.1.1.1 Program Structure

A C program need not all be translated at the same time. [...] After preprocessing, a preprocessing translation unit is called a translation unit. Previously translated translation units may be preserved individually or in libraries. [...] Translation units may be separately translated and then later linked to produce an executable program.

5.1.1.2 Translation Phases

[...]

7. White-space characters separating tokens are no longer significant. Each preprocessing token is converted into a token. The resulting tokens are syntactically and semantically analyzed and translated as a translation unit.

8. All external object and function references are resolved. Library components are linked to satisfy external references to functions and objects not defined in the current translation. All such translator output is collected into a program image which contains information needed for execution in its execution environment.

5.1.2.3 Program Execution

8. EXAMPLE 1 An implementation might define a one-to-one correspondence between abstract and actual semantics: at every sequence point, the values of the actual objects would agree with those specified by the abstract semantics. The keyword volatile would then be redundant.

9. Alternatively, an implementation might perform various optimizations within each translation unit, such that the actual semantics would agree with the abstract semantics only when making function calls across translation unit boundaries. In such an implementation, at the time of each function entry and function return where the calling function and the called function are in different translation units, the values of all externally linked objects and of all objects accessible via pointers therein would agree with the abstract semantics. Furthermore, at the time of each such function entry the values of the parameters of the called function and of all objects accessible via pointers therein would agree with the abstract semantics. In this type of implementation, objects referred to by interrupt service routines activated by the signal function would require explicit specification of volatile storage, as well as other implementation-defined restrictions.

Re: How to zero a buffer

#102

Earlier quoted context omitted.

I'm gonna go ahead and say it: Perhaps in retrospect C is an inappropriate choice of language for these kinds of applications. This "Performance at all costs, including safety and predictability" thing may be appropriate in video games, but for security-critical applications that philosophy is downright negligent.

Dead store elimination is a really important optimization, and it comes out of bog-standard compiler optimizations like SROA on SSA form IR. You really want your compiler to perform it for acceptable performance.

Yes. But dead store elimination combined with being allowed to dereference arbitrary pointers and leaving newly allocated blocks of memory uninitialized is problematic. Contrary to what the C standard would like us to believe, those other features do mean that dead store elimination alters the semantics of a program. It doesn't impact the semantics of the procedure whose dead stores are being eliminated, but it alters the semantics of arbitrary operations elsewhere in the program because it could influence the result I get when I dereference a pointer dereference or examine the contents of a newly-allocated block of memory.

In most cases that distinction is nit-picky. It can be perfectly reasonable for the language to throw up its hands, shout "undefined behavior", and just assume that whatever random uncontrolled thing happens won't be too terrible, assuming whatever your program does isn't too important. But for security-critical applications it's a really stinking important distinction, because the range of possible behaviors found in the "undefined" category includes things like Heartbleed.

Re: How to zero a buffer

#103

Earlier quoted context omitted.

From the article, "Some people will try this with secure_memzero in a separate C file. This will trick yet more compilers, but no guarantees — with link-time optimization the compiler may still discover your treachery." https://gcc.gnu.org/wiki/LinkTimeOptimization http://llvm.org/docs/LinkTimeOptimization.html

1. It is the compiler that is committing treachery here. This stuff stretches, if not outright breaks, the translation model given in the C standard, where it is clear that a program is separated into translation units, and that linkage resolves external names. 2. You bring this on yourself; it's not enabled by default by ordinary optimization options like -O2 or -O3. You have to ask for it, and so you must know what…

People do stupid things sometimes. Yes, I agree you shouldn't apply optimizations to security critical code without fully understanding the ramifications. It is also the case that you shouldn't write security critical code that might break under optimizations, where you can avoid it.

Re: How to zero a buffer

#104

You should have test cases to verify the zeroing behavior in the object code. Even if the standard says a compiler must do something does not mean that it does.

The difficult thing is that any way to verify the zeroing behavior would change the compiler's decision about whether it could elide the call to memset. So it's possible (well, almost guaranteed) that the test would succeed even though the memory wouldn't actually be zeroed in production.

Re: How to zero a buffer

#105
post #55

Earlier quoted context omitted.

Perhaps in retrospect this was an inappropriate choice of definition, at least for cryptographic operations.

I'm gonna go ahead and say it: Perhaps in retrospect C is an inappropriate choice of language for these kinds of applications. This "Performance at all costs, including safety and predictability" thing may be appropriate in video games, but for security-critical applications that philosophy is downright negligent.

C may have pitfalls for "these kinds of applications," but it has some strengths that other languages don't. Since C gives the programmer significant control over memory allocation, it's possible to avoid various kinds of timing attacks related to cache misses (possible is not the same as easy). Many languages don't give the programmer the necessary tools to do that.

Re: How to zero a buffer

#106

Earlier quoted context omitted.

I'm gonna go ahead and say it: Perhaps in retrospect C is an inappropriate choice of language for these kinds of applications. This "Performance at all costs, including safety and predictability" thing may be appropriate in video games, but for security-critical applications that philosophy is downright negligent.

C may have pitfalls for "these kinds of applications," but it has some strengths that other languages don't. Since C gives the programmer significant control over memory allocation, it's possible to avoid various kinds of timing attacks related to cache misses ( possible is not the same as easy ). Many languages don't give the programmer the necessary tools to do that.

There are two different things you can mean when you say "C is not suitable to these kinds of applications". One is the more extreme, "You should not be using C, you should be using because it is more suitable." That's a bit of a hard sell; though specific alternatives should be evaluated on their merits. There is also, "there are design choices that have been made in C that make it worse for these applications than C would have been were it not for those choices", which seems an easy case to make.

Re: How to zero a buffer

#107

Earlier quoted context omitted.

I'm gonna go ahead and say it: Perhaps in retrospect C is an inappropriate choice of language for these kinds of applications. This "Performance at all costs, including safety and predictability" thing may be appropriate in video games, but for security-critical applications that philosophy is downright negligent.

I'm not aware of any language that would be better. Most languages don't even let you touch memory to try to zero it.

There's basically only one (or a family of) languages that have no optimisation at all, and enable complete control over what the machine does - Asm. The code you get is exactly the code you write, no matter how efficient or inefficient it is. This also enables much better the prevention of other attacks like timing/power analysis, since you can effectively insert dummy instructions as needed to keep the timing and power well-behaved.

The biggest downside I see is that it's non-portable, but the reality is that there's not all that many architectures out there to port to anyway (x86, ARM, MIPS probably covers 90%+) and for truly security-critical code having that level of control could be worth it. (This also avoids the "trusting the compiler" problem - an assembler is far easier to verify correctness of than even the simplest C compiler...)

Re: How to zero a buffer

#108
post #68

It is a little mind boggling that support for proper handling of this didn't arrive until c11. For a symmetric cipher without a demanding setup/init phase - would it make sense to just do a few rounds on a buffer using the zeroed key? Obviously quite a few more cycles, but should at least be a predictable (constant) overhead?

What do you mean? The solution that Percival presents compiles fine on my C89 compiler.

Re: How to zero a buffer

#109
Interesting. This appears to solve a more general problem, which is: how to create a barrier against inter-procedural optimization and dead code elimination.

I wonder if this trick could also be used to solve the double-checked locking problem.

From the quintessential DCLP paper (http://www.aristeia.com/Papers/DDJ_Jul_Aug_2004_revised.pdf):

    Consider again the line that initializes pInstance:

    pInstance = new Singleton;

    This statement causes three things to happen:
    Step 1: Allocate memory to hold a Singleton object.
    Step 2: Construct a Singleton object in the allocated memory.
    Step 3: Make pInstance point to the allocated memory.

    [...]

    DCLP will work only if steps 1 and 2 are completed before
    step 3 is performed, but *there is no way to express this
    constraint in C or C++*.
But Colin's pattern here seems to be a way of indeed guaranteeing this. The volatile function pointer is a barrier against inter-procedural optimization: if the function must be called, then step 3 cannot possibly be performed before steps 1 and 2.

(There might still be necessary hardware barriers that are missing, and the lack of a memory model for pre-C11/C++11 probably makes it all technically undefined behavior anyway. But the key sequential ordering constraint that was claimed inexpressible in C and C++ appears to indeed be expressible with this trick, if indeed the trick works for guaranteeing a call to memset).

Re: How to zero a buffer

#110

Earlier quoted context omitted.

I'm not aware of any language that would be better. Most languages don't even let you touch memory to try to zero it.

There's basically only one (or a family of) languages that have no optimisation at all, and enable complete control over what the machine does - Asm. The code you get is exactly the code you write, no matter how efficient or inefficient it is. This also enables much better the prevention of other attacks like timing/power analysis, since you can effectively insert dummy instructions as needed to keep the timing and p…

"There's basically only one (or a family of) languages that have no optimisation at all"

... sort of. Chips themselves perform some optimizations.

Post reply on HN