Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop Memsetting Structures

anmolsarma.in

Post reply on HN
Print view View original

61–70 of 277 posts

Re: Stop Memsetting Structures

#61

Post by hedora »

anyfoo wrote: post #47
wbl wrote: post #41

Earlier quoted context omitted.

Please explain how you send a struct over the network without UB and leaking the padding.

By knowing its exact size with sizeof(), and using that and a cast to uint8_t* to both bzero() it out before use and memcpy()ing it when sending it. Unless the C standard says that the compiler may change the padding bytes of the existing storage?

To be clear, it is something like this, which is not bad at all to write:

  struct st s;
  bzero(&s, sizeof(s));
  // ...
  write(fd, &s, sizeof(s));
I’ve dealt with many systems that have tons of unnecessary serialization logic, and found them slower and buggier than just letting the C spec define the data format.

Having said that, it is possible to get this sort of thing right with C++ and templates (thanks to inlining).

Higher-level languages without compile time code generation (like Java and Python) usually make serialization a train wreck in comparison with “send raw structs” or “make the [C++] compiler generate and typecheck the machine code that does the serialization”

Re: Stop Memsetting Structures

#62

Post by benou »

Frankly, use the style that is idiomatic to your code base is what is the most important. Keeping things consistent is what matters.

And if you start a new project, just use the style you prefer and enforce it :)

Re: Stop Memsetting Structures

#63

Post by loeg »

_bxg1 wrote: post #5
thenewwazoo wrote: post #3

> Cue the Rust Evangelism Strike Force chiming in to say that there is really no reason to be writing new C code in 2019. Even as a member of the Force (RIIR 1st brigade), I laughed pretty heartily at this. :)

Same. But also, I feel like most of The Force's efforts are directed at C++. I could be wrong.

Rust is a big language, like C++. It feels too complicated compared to C.

Re: Stop Memsetting Structures

#64

Post by kabdib »

hedora wrote: post #61
anyfoo wrote: post #47

Earlier quoted context omitted.

By knowing its exact size with sizeof(), and using that and a cast to uint8_t* to both bzero() it out before use and memcpy()ing it when sending it. Unless the C standard says that the compiler may change the padding bytes of the existing storage?

To be clear, it is something like this, which is not bad at all to write: struct st s; bzero(&s, sizeof(s)); // ... write(fd, &s, sizeof(s)); I’ve dealt with many systems that have tons of unnecessary serialization logic, and found them slower and buggier than just letting the C spec define the data format. Having said that, it is possible to get this sort of thing right with C++ and templates (thanks to inlining). H…

There's enough wiggle room in the standard that two compilers would be happy making two compliant but still different physical layouts for the same struct declaration. I'm not even worrying about cross-platform issues or bitfields yet.

You can get away with directly writing a struct for simple structs, but eventually it will bite you.

Re: Stop Memsetting Structures

#65

Post by a_t48 »

mehrdadn wrote: post #60
DaiPlusPlus wrote: post #58

Earlier quoted context omitted.

How does sending raw memory to the socket work if you have a struct filled with pointers?

I think they simply don't do that with structs that contain pointers?

Indeed, this. If you have data with pointers, you either need to flatten (read: copy) your structure into something appropriate for structuring, or just get a proper serialization layer.

Re: Stop Memsetting Structures

#66

Post by loeg »

Someone wrote: post #22
flohofwoe wrote: post #6

C99 designated init doesn't work for C code that also needs to compile in C++ mode (...yet, a limited subset of designated init is coming in C++20, clang also seems to be more relaxed about this and allows the full C99 designated init also in C++ right now). For code that only needs to compile as C, I agree. It's one of the best (if not the best addition) to the C language. A nice addition would be default values for…

My C++ is rusty (no pun intended), but if your code also has to compile as C++, I think this will work instead of memset: struct addrinfo hints = {0}; If C++ requires struct addrinfo hints = {}; instead, you could create a macro, and write struct addrinfo hints = ALL_ZEROES;

Sure. C++ has initializers ({1,2,3}), it just doesn't have C99 designated initializers ({.a = 1, .b = 2, .c = 3}). This is one of the main pain points around C++ not being a true superset of C.

Re: Stop Memsetting Structures

#67

Post by loeg »

cperciva wrote: post #7

Another reason to not memset structures: In the code struct foo { void * bar; } baz; ... memset(&baz, 0, sizeof(struct foo)); assert(baz.bar == NULL); it's possible for the assertion to fail, since NULL it not guaranteed to be represented in memory by a zero bytes.

This is a weak reason. In practice, machines that do not represent NULL as zero no longer exist and new designs of this nature would break so much existing code, it would be difficult for them to gain wide adoption in the marketplace. The C standard hasn't decided to require NULL be represented as zero yet (and may never, to preserve compatibility with machines it used to have defined behavior on), but most C code runs on machines where NULL is represented with zero bits.

(You're absolutely correct that the standard does not require NULL be represented with zero bits, though.)

Re: Stop Memsetting Structures

#68

Post by amboar »

wbl wrote: post #41
anyfoo wrote: post #2

Does that also zero the padding that may exist between elements of the structure (or at its end)? Because if not, not memset'ing will open you up to all kinds of info leaks, especially if you plan to send that struct over the network (or to another process, or from kernel to userspace...). EDIT: Actually I have to apologize, because I’m not certain what the C standard says about whether the padding bytes can change w…

Please explain how you send a struct over the network without UB and leaking the padding.

Use a packed struct (e.g. __attribute__((packed))). You may take a performance hit due to lack of alignment, but that's the judgement call

Re: Stop Memsetting Structures

#69

Post by loeg »

ThrowawayR2 wrote: post #23
cperciva wrote: post #7

Another reason to not memset structures: In the code struct foo { void * bar; } baz; ... memset(&baz, 0, sizeof(struct foo)); assert(baz.bar == NULL); it's possible for the assertion to fail, since NULL it not guaranteed to be represented in memory by a zero bytes.

> it's possible for the assertion to fail, since NULL it not guaranteed to be represented in memory by a zero bytes. Technically correct (which is of course the best kind of correct) but you'd be hard pressed to find a system in 2019 where NULL != (void*)0. The most recent machines with non-zero NULL in the C FAQ entry on the matter ( http://c-faq.com/null/machexamp.html ) date back to the mid '90s.

Nah, NULL is guaranteed to be == (void * )0 by the standard. The allowed divergence is in how (void * )0 is represented as bits in memory, i.e., memcmp(zeroes, (ptr = NULL), sizeof(ptr)) == 0?

Re: Stop Memsetting Structures

#70

Post by loeg »

paavoova wrote: post #49
cperciva wrote: post #7

Another reason to not memset structures: In the code struct foo { void * bar; } baz; ... memset(&baz, 0, sizeof(struct foo)); assert(baz.bar == NULL); it's possible for the assertion to fail, since NULL it not guaranteed to be represented in memory by a zero bytes.

So if you're zeroing out a struct like so struct S {size_t a, void *p;}; foo = (struct S) {0}; Does the standard require the compiler to set any pointers to NULL, or will everything be implicitly zeroed leaving pointers possibly improperly initialized to NULL?

The pointers are required to be initialized to NULL in your example (even if the representation is different from memset of zero).
Post reply on HN