Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop Memsetting Structures

anmolsarma.in

Post reply on HN
Print view View original

101–110 of 277 posts

Re: Stop Memsetting Structures

#101

Post by jfk13 »

klingonopera wrote: post #92
userbinator wrote: post #34

Because it’s 2019! It really irritates me when people give arguments like this. I seriously don't care what year it is, I care that the code I write works. A lot of projects still use C89 because they want the widest portability. C99 isn't very new, but especially in the embedded and other niche industries, C89 (often with some extensions) is all you get. That said, "= { 0 };" is almost always usable instead of a mem…

I just had to google it: https://stackoverflow.com/questions/1912325/checking-for-nul... There is history to this practice. And also, foresight: 1) I may want to redefine free(). 2) If I do, my free() does not null-check. 3) Which is why my main code would null-check. I'm an embedded developer, never do much on application work, and used to do that with C++, so I'm not too familiar with the heap allocation intricacie…

C++ explicitly allows `delete` to be called on a null pointer.

E.g. see https://en.cppreference.com/w/cpp/language/delete

Re: Stop Memsetting Structures

#102

Post by iforgotpassword »

nerdile wrote: post #91

I disagree with this author on both counts. 1. Zero filling is theoretically unnecessary, yes. But theory is not often reality. Zero filling is a defensive measure that protects you from other code that makes unhealthy assumptions. Code external to yours may make assumptions about what lies in your padding, either by doing memcmp's, or adding new fields in a later version of the struct and intending it to be ABI comp…

> Code external to yours may make assumptions about what lies in your padding, either by doing memcmp's, or adding new fields in a later version of the struct and intending it to be ABI compatible between modules.

The content of the padding between members can arbitrarily change during assignments of members, so code relying on the content of it is broken anyways. In the case of an extended struct both methods are equally broken if you don't recompile your module with the updated definition of the struct. Your memset will still have the old size of the struct as the third argument.

Re: Stop Memsetting Structures

#103

Post by hedora »

kabdib wrote: post #64
hedora wrote: post #61

Earlier quoted context omitted.

To be clear, it is something like this, which is not bad at all to write: struct st s; bzero(&s, sizeof(s)); // ... write(fd, &s, sizeof(s)); I’ve dealt with many systems that have tons of unnecessary serialization logic, and found them slower and buggier than just letting the C spec define the data format. Having said that, it is possible to get this sort of thing right with C++ and templates (thanks to inlining). H…

There's enough wiggle room in the standard that two compilers would be happy making two compliant but still different physical layouts for the same struct declaration. I'm not even worrying about cross-platform issues or bitfields yet. You can get away with directly writing a struct for simple structs, but eventually it will bite you.

Packed structs are reasonably standard (even gcc and visual studio can be made to pack in the same way using the same syntax these days).

The remaining issue is endianness, but you can configure ARM to match x86 at boot so shrug.

Also, if I tell any reasonably experienced developer my protocol sends a packed struct with these fields:

  uint8_t version
  uint16_t reserved
  uint8_t op
  int32_t status
  uint32_t data_length
  // void data[]
They’ll be able to write a correct deserializer in pretty much any language, and can probably write it in ~ 10 lines of code.

I agree that packed structs can be tricky, but they’re much better than the vast majority of serialization libraries I’ve dealt with (including all of the open source ones, and I’ve used all the usual suspects).

Re: Stop Memsetting Structures

#104

Post by _kst_ »

hedora wrote: post #61
anyfoo wrote: post #47

Earlier quoted context omitted.

By knowing its exact size with sizeof(), and using that and a cast to uint8_t* to both bzero() it out before use and memcpy()ing it when sending it. Unless the C standard says that the compiler may change the padding bytes of the existing storage?

To be clear, it is something like this, which is not bad at all to write: struct st s; bzero(&s, sizeof(s)); // ... write(fd, &s, sizeof(s)); I’ve dealt with many systems that have tons of unnecessary serialization logic, and found them slower and buggier than just letting the C spec define the data format. Having said that, it is possible to get this sort of thing right with C++ and templates (thanks to inlining). H…

A small quibble: bzero() is not defined by the C standard. memset() is.

Re: Stop Memsetting Structures

#105

Post by hedora »

iforgotpassword wrote: post #102
nerdile wrote: post #91

I disagree with this author on both counts. 1. Zero filling is theoretically unnecessary, yes. But theory is not often reality. Zero filling is a defensive measure that protects you from other code that makes unhealthy assumptions. Code external to yours may make assumptions about what lies in your padding, either by doing memcmp's, or adding new fields in a later version of the struct and intending it to be ABI comp…

> Code external to yours may make assumptions about what lies in your padding, either by doing memcmp's, or adding new fields in a later version of the struct and intending it to be ABI compatible between modules. The content of the padding between members can arbitrarily change during assignments of members, so code relying on the content of it is broken anyways. In the case of an extended struct both methods are eq…

If you use:

  sizeof(struct_instance)
the compiler will automatically update the size passed to memcpy. (I like passing the instance to sizeof and not the type, just in case the type of the thing being copied changes in a future version of the code.)

Re: Stop Memsetting Structures

#106

Post by a_t48 »

repsilat wrote: post #93
a_t48 wrote: post #65

Earlier quoted context omitted.

Indeed, this. If you have data with pointers, you either need to flatten (read: copy) your structure into something appropriate for structuring, or just get a proper serialization layer.

I guess there could be some fun workarounds with ptrdiff_t instead of pointers. "My child is 1000 bytes that way in memory" etc.

That's a pretty classical file serialization trick to pack a bunch of heterogeneous data in one file. A header/manifest starting the file, with an array of names, offsets, and sizes.

Re: Stop Memsetting Structures

#107

Post by klingonopera »

userbinator wrote: post #99
klingonopera wrote: post #92

Earlier quoted context omitted.

I just had to google it: https://stackoverflow.com/questions/1912325/checking-for-nul... There is history to this practice. And also, foresight: 1) I may want to redefine free(). 2) If I do, my free() does not null-check. 3) Which is why my main code would null-check. I'm an embedded developer, never do much on application work, and used to do that with C++, so I'm not too familiar with the heap allocation intricacie…

I may want to redefine free(). 2) If I do, my free() does not null-check. I think redefining standard functions to do something very unexpected is very poor decision, especially when it means you now have to burden all its callsites with an extra check. But I'm quite sure that "delete" in C++ e.g. definitely needs to be null-checked (unless they changed that in some newer C++ revision No, it was never needed in C++ e…

Yes, you're right about delete. I still do it, and will still do it, though. I have my reasons.

No, I wouldn't redefine free(), I'd use my own terminology, but I'd search the code for it and replace it verbatim, and seeing that it apparently is standard practice for both C++ and C to null-check their pointers before deletion, I have all the more reason to do so, because my code assumed that it didn't.

Delete shouldn't null check, it should delete memory at location, no matter what address. It should be up to the implementation to check that pointer is valid.

EDIT: I should maybe add that the next line of code is in my case almost always "pointer = NULL" following "free(pointer)". The null-check is not just a technicality of checking of pointer-validity ("Pointer not null?"), but also code-syntactics ("Has this item not been deleted yet?").

Re: Stop Memsetting Structures

#108

Post by rocqua »

_bxg1 wrote: post #100
loeg wrote: post #63

Earlier quoted context omitted.

Rust is a big language, like C++. It feels too complicated compared to C.

I've only written a little of each, but my impression is that C has mostly remained a focused, effective language for careful systems programming. While C++ has become a sprawling cthulhu of leaky abstractions and other nasty foot-guns and people only use it because "it's as fast as C but sorta feels like a higher-level language". And that therefore Rust is a strict improvement over C++, but not necessarily C.

Honestly I tend to use c++ as C with easy strings for user feedback.

Re: Stop Memsetting Structures

#110

Post by hedora »

_bxg1 wrote: post #100
loeg wrote: post #63

Earlier quoted context omitted.

Rust is a big language, like C++. It feels too complicated compared to C.

I've only written a little of each, but my impression is that C has mostly remained a focused, effective language for careful systems programming. While C++ has become a sprawling cthulhu of leaky abstractions and other nasty foot-guns and people only use it because "it's as fast as C but sorta feels like a higher-level language". And that therefore Rust is a strict improvement over C++, but not necessarily C.

I thought this way too until I worked with competent C++ devs on a modern code base. (By my definition of “competent,” no one ever encountered by the Rust brigade bloggers is competent, fwiw).

At some point, I realized that C++ is a language for implementing new languages in a way that is backward compatible with existing programs.

Templates are Turing complete (lazily evaluated, purely functional, and side-effect free), so whatever your language of choice can do can be backported to C++ (granted, painfully, but probably less painfully than throwing out all your legacy code and starting from scratch).

On top of this, C++ supports zero-cost abstraction, so it will get very close to low level C performance (including constant propagation, loop unrolling, type-unsafe transformations, hoisting, inlining, etc, etc) even if you throw piles of lambdas and encapsulated methods at it.

The costs, of course, are that the compiler is slooooowww, and you inherit all the memory safety warts of C by default. (Emphasis on “by default”, because memory safe subsets of C++ are perfectly workable.)

Post reply on HN