Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop Memsetting Structures

anmolsarma.in

Post reply on HN
Print view View original

161–170 of 277 posts

Re: Stop Memsetting Structures

#161

Post by cardiffspaceman »

jdblair wrote: post #134
anyfoo wrote: post #2

Does that also zero the padding that may exist between elements of the structure (or at its end)? Because if not, not memset'ing will open you up to all kinds of info leaks, especially if you plan to send that struct over the network (or to another process, or from kernel to userspace...). EDIT: Actually I have to apologize, because I’m not certain what the C standard says about whether the padding bytes can change w…

But if you're slamming in-memory structs over the wire, you're using packed structs. You explicitly don't want padding when you use structs that way. Whether you should do this in 2019 is another question, but sometimes it's still the easiest way to implement a binary protocol.

And you're not writing code for big-endian machines to interoperate with little-endian machines.

It's easy to write binary protocol code that way but it isn't portable.

Re: Stop Memsetting Structures

#162

Post by blattimwind »

hedora wrote: post #103
kabdib wrote: post #64

Earlier quoted context omitted.

There's enough wiggle room in the standard that two compilers would be happy making two compliant but still different physical layouts for the same struct declaration. I'm not even worrying about cross-platform issues or bitfields yet. You can get away with directly writing a struct for simple structs, but eventually it will bite you.

Packed structs are reasonably standard (even gcc and visual studio can be made to pack in the same way using the same syntax these days). The remaining issue is endianness, but you can configure ARM to match x86 at boot so shrug . Also, if I tell any reasonably experienced developer my protocol sends a packed struct with these fields: uint8_t version uint16_t reserved uint8_t op int32_t status uint32_t data_length //…

> The remaining issue is endianness, but you can configure ARM to match x86 at boot so shrug.

ARM in BE mode is pretty rare / nonexistent. Small MIPS chips still float around in mostly low-end networking gear, though, and those do tend to run in BE.

Re: Stop Memsetting Structures

#163

Post by userbinator »

deathanatos wrote: post #56
userbinator wrote: post #34

Because it’s 2019! It really irritates me when people give arguments like this. I seriously don't care what year it is, I care that the code I write works. A lot of projects still use C89 because they want the widest portability. C99 isn't very new, but especially in the embedded and other niche industries, C89 (often with some extensions) is all you get. That said, "= { 0 };" is almost always usable instead of a mem…

> but especially in the embedded and other niche industries, C89 (often with some extensions) is all you get. Embedded and niche industries are forcing their programmers to work with 30 year old tooling? (Instead of 20 year tooling?) I think at some point, writing yourself a transpiler from C99 to C89 becomes profitable… (The JS community did it for a lot less than that w/ babel…)

Embedded and niche industries are forcing their programmers to work with 30 year old tooling? (Instead of 20 year tooling?)

Why does how old it is mean anything at all? I personally choose C89 because of its wider compatibility, and I don't think I'm any worse off because of it. As the saying goes, "it's not about the tool, but how you use it." Incidentally, the best programmers I've worked with have also been the ones with the most conservative choice of tools.

I think at some point, writing yourself a transpiler from C99 to C89 becomes profitable… (The JS community did it for a lot less than that w/ babel…)

If anything, I think the JS community could learn a lot from the "C community" (a very loosely defined term...) --- the huge amount of churn with anything related to JS and web stuff in general turns me off.

Re: Stop Memsetting Structures

#164

Post by NikkiA »

jschwartzi wrote: post #4

I'll stop memsetting structures as soon as you support designated initializer lists in C++.

Apparently it's in C++20, so feel free to stop memsetting whenever you update your C++ compiler next (clang needs a flag to enable it, gcc has had it as part of it's c++2a support for 2 years)

Re: Stop Memsetting Structures

#165

Post by ploxiln »

mehrdadn wrote: post #48
unmole wrote: post #45

Earlier quoted context omitted.

I am aware of the fact that unitilized padding can have nasty side effects: https://lwn.net/Articles/417989/ But yeah, I should have mentioned the caveats too. I will update the post once I'm back at my computer.

I'm confused what I'm supposed to look for in that article (do you mean you were the author?) but anyway -- I feel like calling this merely a "caveat" gives the wrong impression? It seems a bit like telling people to mix bleach with vinegar and then saying "Oops, sorry, I did know that it produces chlorine! I forgot to mention that caveat." The caveat isn't just a side note for the margins, it's a critical reason why…

Failing to clear padding in structs is _really_ not often a problem. It's pretty much only a problem when copying un-packed structs from kernel-space to user-space.

So if you're not working on an OS kernel somewhere in the syscall path - really not a common activity even among C programmers - it's not a problem. If you are passing a struct from one part of your program to another part, or to a library you use which is not isolated or sandboxed in some way, it's not a problem. If you pass a struct to a _more_ privileged context like a syscall, it's not a problem. If you are passing structs over a network connection or writing to a file, you'll pack them and be aware of every byte anyway, in order to be compatible with another implementation reading them.

If you _are_ passing a struct from a more privileged context to a less privileged context, then yeah you have to memset().

Re: Stop Memsetting Structures

#166

Post by hermitdev »

hermitdev wrote: post #129
dataflow wrote: post #14

Earlier quoted context omitted.

Nope it doesn't: //GCC: -O0 //MSVC: /Od #include #include struct S { unsigned char x; int y; }; int f(int f) { S a; memset(&a, f, sizeof(a)); unsigned char y[2]; memcpy(&y, &a, sizeof(y)); return y[1]; } int g() { S a = { 1, 2 }; unsigned char y[2]; memcpy(&y, &a, sizeof(y)); return y[1]; } int main(int argc, char *argv[]) { f(0xDD); printf("%#x\n", g()); f(0xFF); printf("%#x\n", g()); } It definitely seems like the…

If you're relying upon the value of padding, you're already into undefined behavior.

If you're either using via either assigning or reading the value that is stored in the padded area of a class or struct you are in undefined behavior, any version of the standard you choose. So, care to explain the downvotes?

Re: Stop Memsetting Structures

#167

Post by mehrdadn »

ploxiln wrote: post #165
mehrdadn wrote: post #48

Earlier quoted context omitted.

I'm confused what I'm supposed to look for in that article (do you mean you were the author?) but anyway -- I feel like calling this merely a "caveat" gives the wrong impression? It seems a bit like telling people to mix bleach with vinegar and then saying "Oops, sorry, I did know that it produces chlorine! I forgot to mention that caveat." The caveat isn't just a side note for the margins, it's a critical reason why…

Failing to clear padding in structs is _really_ not often a problem. It's pretty much only a problem when copying un-packed structs from kernel-space to user-space. So if you're not working on an OS kernel somewhere in the syscall path - really not a common activity even among C programmers - it's not a problem. If you are passing a struct from one part of your program to another part, or to a library you use which i…

I agree, but the frequency isn't the issue. You could say that's where my analogy breaks down, and if that's your point then maybe it does (although how often do people want to mix bleach with vinegar?), but that entirely misses the point. The point was that the failure mode, when it does come up, is a critical one, and if you're going to criticize a safe coding practice and tell people to switch to a potentially dangerous one, it behooves you to explain to them this very fact. It's just irresponsible to tell people to switch to a dangerous practice without making them aware of that fact and telling them how to deal with it.

Re: Stop Memsetting Structures

#168

Post by jasonhansel »

hedora wrote: post #110
_bxg1 wrote: post #100

Earlier quoted context omitted.

I've only written a little of each, but my impression is that C has mostly remained a focused, effective language for careful systems programming. While C++ has become a sprawling cthulhu of leaky abstractions and other nasty foot-guns and people only use it because "it's as fast as C but sorta feels like a higher-level language". And that therefore Rust is a strict improvement over C++, but not necessarily C.

I thought this way too until I worked with competent C++ devs on a modern code base. (By my definition of “competent,” no one ever encountered by the Rust brigade bloggers is competent, fwiw). At some point, I realized that C++ is a language for implementing new languages in a way that is backward compatible with existing programs. Templates are Turing complete (lazily evaluated, purely functional, and side-effect fr…

The problem is that (in my experience) it's very hard to reach that level of competence. I certainly haven't, so I feel safer with Rust.

Re: Stop Memsetting Structures

#169

Post by cperciva »

tedunangst wrote: post #141
cperciva wrote: post #136

Earlier quoted context omitted.

Yes. If you look at the commit history you'll find that we've fixed a few of them already.

Yeah, I saw the comment and big block of bulls in bad tar. Here's two from memory: parent pointer in tree_entry, although it seems unused anyway. buff pointer in the link hash thing. (My bad, I looked at it on my laptop, which is not here anymore. I can drop an email later.)

Thanks! I'll try to remember about this when I'm back at my laptop but an email will make sure I don't forget.

FWIW I'm less strict about standards compliance in the libarchive-derived code, since that frobs lots of unportable bits anyway.

Re: Stop Memsetting Structures

#170

Post by MiroF »

_bxg1 wrote: post #5
thenewwazoo wrote: post #3

> Cue the Rust Evangelism Strike Force chiming in to say that there is really no reason to be writing new C code in 2019. Even as a member of the Force (RIIR 1st brigade), I laughed pretty heartily at this. :)

Same. But also, I feel like most of The Force's efforts are directed at C++. I could be wrong.

That makes no sense - C++ eliminates many of the memsafety concerns that are problematic in C
Post reply on HN