Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop Memsetting Structures

anmolsarma.in

Post reply on HN
Print view View original

191–200 of 277 posts

Re: Stop Memsetting Structures

#191

Post by saagarjha »

loeg wrote: post #67
cperciva wrote: post #7

Another reason to not memset structures: In the code struct foo { void * bar; } baz; ... memset(&baz, 0, sizeof(struct foo)); assert(baz.bar == NULL); it's possible for the assertion to fail, since NULL it not guaranteed to be represented in memory by a zero bytes.

This is a weak reason. In practice, machines that do not represent NULL as zero no longer exist and new designs of this nature would break so much existing code, it would be difficult for them to gain wide adoption in the marketplace. The C standard hasn't decided to require NULL be represented as zero yet (and may never, to preserve compatibility with machines it used to have defined behavior on), but most C code ru…

I feel like a compiler is within its rights to optimize out that check if it so wishes, though.

Re: Stop Memsetting Structures

#192

Post by saagarjha »

HorstG wrote: post #8

The article gives very dangerous advice, especially for networking code. Padding will not be overwritten and when transmitting the struct (provided there is no attribute packed or the compiler ignores it) information will leak. Yes, someone will point out that there are languages that do not have such problems. However, when still using C, one has to be extremely careful...

I am reasonably certain that any code that happens to transmit padding bytes (or observe them in any way) is abusing at the very least implementation-defined behavior.

Re: Stop Memsetting Structures

#193

Post by saagarjha »

MichaelMoser123 wrote: post #143
olliej wrote: post #75

Earlier quoted context omitted.

Nope, memset will often be “optimized” such that padding is not zeroed. It’s not because the compiler sees that padding isn’t observed, it’s because reading from padding is undefined. Because reading from padding is undefined the write (also undefined) by definition has no observable side effects. Memset_s is the only way to guarantee that padding is actually zeroed, well memset or bzero_s obviously. It’s critically…

are you sure that the padding is not zeroed by memset? [1] says that memset_s exists because it is not optimized away if it just clears a variable that will not be used after the call (unlike memset) On the other end: people who do this for security reasons say not to trust memset_s is 'an optional feature of c11 and not really portable' [2] [1] https://en.cppreference.com/w/c/string/byte/memset [2] https://www.crypt…

I don't think memset is required to zero out any padding, because the effects of this are not observable.

Re: Stop Memsetting Structures

#194

Post by saagarjha »

olliej wrote: post #94
eridius wrote: post #77

Earlier quoted context omitted.

Right, but zeroing padding only matters for security reasons. If your memset isn't being done for security reasons, then memset() is better than memset_s().

I just realized you might be thinking of security in the sense of keys/partial keys. Things that matter: * vtable and code pointers: leaks the aslr slide * data pointers: leak heap location * type ids: can be used to spoof objects * length and size fields: can give you information about heap layout Etc etc. All of these get attacked.

The standard (and compilers that conform to it) have no sense of security baked into them. They don't know that they should wipe certain buffers because as far as it's concerned the C "abstract virtual machine" provides no way of accessing such data. Of course, as a real software engineer you know that you will make mistakes and these will frequently allow implementation details to leak in ways that are exploitable; however, there is no standards-compliant way to protect against this because as far as it is concerned you always write correct code. There are ways to influence the compiler to do certain things that help in the case of mistakes (memset_s, __attribute__((packed))) but these do not and cannot solve the underlying problem because they are by definition non-portable: they change things that are implementation-defined and not standards-defined.

Re: Stop Memsetting Structures

#195

Post by saagarjha »

olliej wrote: post #128
Matheus28 wrote: post #115

Earlier quoted context omitted.

You shouldn't be sending structs over the network, since their specific layout depends on the compiler (and version). At least make them packed if you're gonna do undefined behavior... But you should just do it the right way and have a wrapper that deals with endianness differences and such: struct S my_struct; // ... push_int64(&output_stream, &my_struct.value1); push_string(&output_stream, my_struct.value2); push_i…

The problem is not sending structs across the network. The problem is processing any data from the network. You have to assume that data is malicious, and will attack any weakness. I’m not talking about serialisation at all - I agreed entirely that sending struct padding across the net is less than optimal :)

> You have to assume that data is malicious, and will attack any weakness.

The point is that the standard cannot help you here because malicious data attacks things outside the scope of the standard.

Re: Stop Memsetting Structures

#196

Post by rurban »

blattimwind wrote: post #154
johncolanduoni wrote: post #123

Earlier quoted context omitted.

This kind of issue happens all the time with structs that a kernel, driver etc. expose to userspace: https://j00ru.vexillium.org/papers/2018/bochspwn_reloaded.pd... Not sending the structure over the network is insufficient, and rigorous memset is a clear solution to this problem. Also, C structure layout for a specific ABI is strictly determined; if that weren’t the case you couldn’t compile two C compilation units…

Also don't expect memset() to actually be a library call at all.

And most importantly dont expect memset to be executed at all. Most -O2 compilers happily optimize the memset away if if doesn't care about the sideeffects. Therefore there exists the "secure" memset variants, which are the insecure variants adding a simple compiler barrier, so it isn't optimized away.

And there exists the very few really secure memset_s variants with a real memory barrier which guarantees memset to be executed in load and store order, flushing the cache, so that Spectre attacks are mitigated. security-relevant crypto libraries, like openssl and variants don't care about that, because memory barriers are "too slow". only the Linux kernel does it properly. It's still a mess.

Re: Stop Memsetting Structures

#197

Post by rurban »

MichaelMoser123 wrote: post #143
olliej wrote: post #75

Earlier quoted context omitted.

Nope, memset will often be “optimized” such that padding is not zeroed. It’s not because the compiler sees that padding isn’t observed, it’s because reading from padding is undefined. Because reading from padding is undefined the write (also undefined) by definition has no observable side effects. Memset_s is the only way to guarantee that padding is actually zeroed, well memset or bzero_s obviously. It’s critically…

are you sure that the padding is not zeroed by memset? [1] says that memset_s exists because it is not optimized away if it just clears a variable that will not be used after the call (unlike memset) On the other end: people who do this for security reasons say not to trust memset_s is 'an optional feature of c11 and not really portable' [2] [1] https://en.cppreference.com/w/c/string/byte/memset [2] https://www.crypt…

This [2] is a dangerous post from a security guy, describing memset_s with just a compiler barrier as "secure". In the age of Spectre only a memset_s with a memory barrier is secure, and currently all memset_s but my safeclib memset_s are insecure. Esp. the ones with "Secure" in its name.

Re: Stop Memsetting Structures

#198

Post by Dylan16807 »

boulos wrote: post #159
Dylan16807 wrote: post #156

Earlier quoted context omitted.

Technically I don't think memset forces the compiler to clear the padding and keep it clear either though. The magic of the C abstract machine.

Do you mean in the sense that a compiler can say "I know what memset is supposed to do" then decides that even though sizeof returns a particular size, that it might ignore that and set fewer bytes when it does a substitution for the memcpy call?

Yes. As long as the behavior observable by a conforming C program is the same, the compiler is allowed to change anything else.

Even if memset in a vacuum is guaranteed, look at the quote by anyfoo. "When a value is stored in an object of structure or union type, including in a member object, the bytes of the object representation that correspond to any padding bytes take unspecified values."

If the compiler knows you're going to write to the members right after the memset, it can put the padding back to its previous values. And by that I mean as far as your code can tell it put it back, but in actuality it never zeroed the padding to begin with.

And since there are performance benefits to pretending, now you have a situation where a "helpful" compiler and a malicious compiler have the same effect: data can get leaked.

Re: Stop Memsetting Structures

#199

Post by swiley »

Ugh, what do you do when the structure definition is updated and you now have uninitialized values?

If there's no explicit initializer I think memset,calloc or ={0} is just fine thanks.

Re: Stop Memsetting Structures

#200

Post by FrozenVoid »

rurban wrote: post #196
blattimwind wrote: post #154

Earlier quoted context omitted.

Also don't expect memset() to actually be a library call at all.

And most importantly dont expect memset to be executed at all. Most -O2 compilers happily optimize the memset away if if doesn't care about the sideeffects. Therefore there exists the "secure" memset variants, which are the insecure variants adding a simple compiler barrier, so it isn't optimized away. And there exists the very few really secure memset_s variants with a real memory barrier which guarantees memset to…

you can write your own memset that bypasses all of that. Its just 2 lines of code.
Post reply on HN