Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop Memsetting Structures

anmolsarma.in

Post reply on HN
Print view View original

241–250 of 277 posts

Re: Stop Memsetting Structures

#241

Post by tom_mellior »

hedora wrote: post #61
anyfoo wrote: post #47

Earlier quoted context omitted.

By knowing its exact size with sizeof(), and using that and a cast to uint8_t* to both bzero() it out before use and memcpy()ing it when sending it. Unless the C standard says that the compiler may change the padding bytes of the existing storage?

To be clear, it is something like this, which is not bad at all to write: struct st s; bzero(&s, sizeof(s)); // ... write(fd, &s, sizeof(s)); I’ve dealt with many systems that have tons of unnecessary serialization logic, and found them slower and buggier than just letting the C spec define the data format. Having said that, it is possible to get this sort of thing right with C++ and templates (thanks to inlining). H…

In the "..." part you presumably meant to include writes to members of s, and those writes are allowed to change padding bytes and leak information.

Re: Stop Memsetting Structures

#242

Post by tom_mellior »

wbl wrote: post #41
anyfoo wrote: post #2

Does that also zero the padding that may exist between elements of the structure (or at its end)? Because if not, not memset'ing will open you up to all kinds of info leaks, especially if you plan to send that struct over the network (or to another process, or from kernel to userspace...). EDIT: Actually I have to apologize, because I’m not certain what the C standard says about whether the padding bytes can change w…

Please explain how you send a struct over the network without UB and leaking the padding.

All answers to this must be wrong as it's simply not possible to do it. You need to take endianness of each field into account, and you cannot do that by treating the struct as a single blob.

Re: Stop Memsetting Structures

#243

Post by nkurz »

jcr1488 wrote: post #238
saagarjha wrote: post #232

Earlier quoted context omitted.

A conforming C compiler can remove a memset that has no side effects.

You mean it can remove a memset() that doesn't cause the observable behaviour to change? For the sake of argument, can you show me some example code where it would be conforming to remove a memset() call? Preferably a realistic example and not a Google'd copypasta. Because it's all too easy to just regurgitate things you heard and think you understood, but no so easy to demonstrate it yourself.

I agree that it would be rare for a memset() used for initialization to be removed, but this is a good recent paper that gives lots of examples of when stores are eliminated in practice:

Dead Store Elimination (Still) Considered Harmful

https://cseweb.ucsd.edu/~klevchen/yjoll-usesec17.pdf

Re: Stop Memsetting Structures

#244

Post by DaiPlusPlus »

a_t48 wrote: post #175
blattimwind wrote: post #155

Earlier quoted context omitted.

Actually works somewhat well for mmap'ed access, too, by using a macro or function to "dereference" "pointers" (offsets).

Or even an OffsetPtr class with overloaded arrow and dereference operator, if you swing that way. :)

What’s the runtime performance of that in-practice? Does x86, x64 and ARM have support for relative offset pointers without any performance impact? And do compilers use them?

Re: Stop Memsetting Structures

#245

Post by dooglius »

saagarjha wrote: post #233
GuB-42 wrote: post #227

It is C99 code. It is worth noting that many C projects go for extreme portability. That's why C is used in the first place, and sometimes C99 is too much. I still sometimes work with compilers that barely support C89. That's for the aerospace industry BTW. There are even some libc that don't accept free(NULL). If you know the platform you are writing for is not ancient, that's fine but if you are writing portable co…

> There are even some libc that don't accept free(NULL). That’s pretty much the completely opposite of portable, since it violates the C standard.

The "extreme portability" he's talking about is portability of projects to compilers and libc implementations that behave badly, possibly in standards-violating ways like this.

Re: Stop Memsetting Structures

#247

Post by klingonopera »

saagarjha wrote: post #183
klingonopera wrote: post #173

Earlier quoted context omitted.

Would this work struct S {char data, char pad0, char pad1, char pad2}; int main() {S dataStruct; printf("%s", dataStruct.data); return 0;} and give me a 4-byte struct or would the compiler optimize it all away and leave me with a 1-byte struct?

Since you don't actually access the size of the structure, I see no reason why it matters. Also, FWIW, your code has undefined behavior because you call printf with the wrong type.

Yes on the UB, I'm passing a "char" into a "char *", sorry about that. I don't pass lone char data that often.

Well, I chose this example specifically to ask whether the resulting memory structure, completely independent of calling a sizeof(), would still be as if I were to call a sizeof() (which then I believe would be 4 bytes large) if not accessing the total size of the struct or any of the padding members.

It makes a difference in the memory footprint, which can become important, if you're programming a device with just 64-bytes total RAM.

Re: Stop Memsetting Structures

#248

Post by _bxg1 »

saagarjha wrote: post #189
hedora wrote: post #110

Earlier quoted context omitted.

I thought this way too until I worked with competent C++ devs on a modern code base. (By my definition of “competent,” no one ever encountered by the Rust brigade bloggers is competent, fwiw). At some point, I realized that C++ is a language for implementing new languages in a way that is backward compatible with existing programs. Templates are Turing complete (lazily evaluated, purely functional, and side-effect fr…

> Templates are Turing complete (lazily evaluated, purely functional, and side-effect free), so whatever your language of choice can do can be backported to C++ (granted, painfully, but probably less painfully than throwing out all your legacy code and starting from scratch). This does not logically follow, since templates are evaluated at compile time and hence have restrictions on what they can do. > C++ supports z…

"since templates are evaluated at compile time and hence have restrictions on what they can do"

This statement doesn't make any sense. A Lisp program can be fully converted to run at compile time; it'll just be lacking in I/O opportunities.

Re: Stop Memsetting Structures

#249

Post by klingonopera »

shaklee3 wrote: post #176
klingonopera wrote: post #107

Earlier quoted context omitted.

Yes, you're right about delete. I still do it, and will still do it, though. I have my reasons. No, I wouldn't redefine free(), I'd use my own terminology, but I'd search the code for it and replace it verbatim, and seeing that it apparently is standard practice for both C++ and C to null-check their pointers before deletion, I have all the more reason to do so, because my code assumed that it didn't. Delete shouldn'…

What are your reasons?

They're in the edit. Precedence of code-flow indicators (I said "code-syntactics" in my original quote, that's wrong, sorry) are more important to me, than keeping away redundancies.

What I am advocating for, is "Before you work on a pointer, check its validity", regardless of what follows after. It's just a general guideline, do it, even if you think it's senseless and redundant. Ever since I've adopted those basics, I've virtually reduced my problems with pointers to zero.

There's danger in accidentally not null-checking, but none in accidentally double-checking. Therefore, the implementation should take care of this (too), or at least consider implications.

I can't really take risks there.

The original author of the article is asking me to "stop" this, which could put a lot of things at stake. No, thank you.

Re: Stop Memsetting Structures

#250

Post by kazinator »

ploxiln wrote: post #165
mehrdadn wrote: post #48

Earlier quoted context omitted.

I'm confused what I'm supposed to look for in that article (do you mean you were the author?) but anyway -- I feel like calling this merely a "caveat" gives the wrong impression? It seems a bit like telling people to mix bleach with vinegar and then saying "Oops, sorry, I did know that it produces chlorine! I forgot to mention that caveat." The caveat isn't just a side note for the margins, it's a critical reason why…

Failing to clear padding in structs is _really_ not often a problem. It's pretty much only a problem when copying un-packed structs from kernel-space to user-space. So if you're not working on an OS kernel somewhere in the syscall path - really not a common activity even among C programmers - it's not a problem. If you are passing a struct from one part of your program to another part, or to a library you use which i…

> If you pass a struct to a _more_ privileged context like a syscall, it's not a problem.

Simply not true; that context could pass a copy of that structure to a less privileged context other than the original one, without doing anything about the padding.

(Well, you could argue that it's the privileged kernel's fault: it should meticulously floss the structure between the members to clear the padding to zero while preserving the values.)

Someone upthread mentioned "sending over a network": that's an example. Sending over a network (or local socket/pipe) begins with pushing it down to a more privileged context, which then assiduously places all the bytes into a buffer that is blasted out on the wire. In this case, the more privileged context has no idea what the structure even is; it's just a blob of bytes.

Post reply on HN