Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Lost Art of C Structure Packing (2014)

catb.org

Post reply on HN
Print view View original

31–40 of 116 posts

Re: The Lost Art of C Structure Packing (2014)

#31

Post by kazinator »

TL;DR: to get smaller structures, don't do stuff like this:

   struct foo {
     char a;
     int b;
     char c;
     int d;
   };
but this:

   struct foo {
     int b;  // or int b, c;
     int c;
     char a;
     char c;
   };
Basically if you sort the types by size in reverse descending order, you get optimal packing without messing with compiler-specific packing extensions that skew alignment and possibly bloat code.

The worst that you will get is padding at the end of the structure so that if two or more of them are arrayed, the first member is correctly aligned at all the array indices.

Re: The Lost Art of C Structure Packing (2014)

#32

Post by cheez »

In this thread: people who do this for a living making people who don't feel dumb.

Guys, it's called "The Lost Art of C Structure Packing" for a reason. Python and Ruby guys have no idea.

Even most C or C++ devs wouldn't know.

Take a chill pill.

Re: The Lost Art of C Structure Packing (2014)

#33

Post by qwertyuiop924 »

Artlav wrote: post #29

Huh? "Lost art"? That's, like, programming 101 grade material, not some "lost art". Then again, i worked in telecom and HPC, and play with uCs, so perhaps i'm wearing the wrong googles...

you are. It's not exactly a lost art, but it's not known except to those inside of the secret, uninviting inner circles of C programming, which you can only enter if you memorize all the UB.

Re: The Lost Art of C Structure Packing (2014)

#34

Post by tedunangst »

cheez wrote: post #32

In this thread: people who do this for a living making people who don't feel dumb. Guys, it's called "The Lost Art of C Structure Packing" for a reason. Python and Ruby guys have no idea. Even most C or C++ devs wouldn't know. Take a chill pill.

That doesn't make it a lost art. I've no clue how to make wine (something to do with grapes?) but I don't describe it as a lost art.

Re: The Lost Art of C Structure Packing (2014)

#35

Post by bluGill »

sfrailsdev wrote: post #19

My understanding (and i I'm wrong, tell me) is that if you try to serialize or save to disk something with a bitfield directly, you can have big problems.

Bitfields orders are "implementation defined". So you need to ask the vendor of every compiler you support what the compiler does. Once you know what the compilers you support do you can write code specific for them and everything works great. It is only when someone tried a different compiler (including an upgrade of the existing one) that you can run into problems.

Re: The Lost Art of C Structure Packing (2014)

#36

Post by qwertyuiop924 »

ESR is a good programmer, and this is a handy guide for those who don't know how to pack structures. He's also insane, so don't trust anything he says.

Oh, you don't think he's insane? He thinks there's a conspiracy amongst women in open source to discredit Linus Torvalds. No, I'm not joking. I wish I was.

Re: The Lost Art of C Structure Packing (2014)

#37

Post by cheez »

tedunangst wrote: post #34
cheez wrote: post #32

In this thread: people who do this for a living making people who don't feel dumb. Guys, it's called "The Lost Art of C Structure Packing" for a reason. Python and Ruby guys have no idea. Even most C or C++ devs wouldn't know. Take a chill pill.

That doesn't make it a lost art. I've no clue how to make wine (something to do with grapes?) but I don't describe it as a lost art.

Most programmers would have known this a few years ago. Programmers today don't. So, it is a lost art.

Re: The Lost Art of C Structure Packing (2014)

#38

Post by vonmoltke »

kazinator wrote: post #31

TL;DR: to get smaller structures, don't do stuff like this: struct foo { char a; int b; char c; int d; }; but this: struct foo { int b; // or int b, c; int c; char a; char c; }; Basically if you sort the types by size in reverse descending order, you get optimal packing without messing with compiler-specific packing extensions that skew alignment and possibly bloat code. The worst that you will get is padding at the…

> The worst that you will get is padding at the end of the structure so that if two or more of them are arrayed, the first member is correctly aligned at all the array indices.

This is not guaranteed. If you want to ensure all array member starting addresses are aligned for all compilers and platforms you need to add explicit dead space. You should also be using the fixed-width numeric types.

Re: The Lost Art of C Structure Packing (2014)

#39

Post by minipci1321 »

bluGill wrote: post #30
minipci1321 wrote: post #5

Sigh... apparently this is now enough of rocket science to be considered for posting on HN... People using packing pragma of GCC should also beware -- an access to a field of a packed structure will be done bytewise, whether a variable happens to be actually aligned or not (I guess the compiler simplified its life by assuming no variable is ever aligned in packet structs), so memory size would go down but CPU use mig…

True (for some processors), but if you are doing packaged structs that should imply you have a reason to tell the compiler you know better than it how to write the struct. That most often (in my experience) means the struct is used for data interchange (ie network or files) and so you have to pay that price for byte access because byte access is a requirement. The other time to do that you might do this is when your…

> but if you are doing packaged structs that should imply you have a reason to tell the compiler you know better

I am pretty sure this could have been implemented better -- for malloc-ed and statically-allocated structs the compiler can know the alignment of the structure, so should be able to emit optimal code according to each case. Admittedly, probably a very minor improvement.

It is the feeling of surprise at the time that made me react -- this side-effect wasn't documented in GCC manual at the time, and still isn't.

We now have the attribute "aligned", so I am not sure what the code would look like for a struct {} __attribute__((__packed__,__aligned(8))).

Re: The Lost Art of C Structure Packing (2014)

#40

Post by ThatGeoGuy »

Although I somewhat tire of constant comparisons between Rust and C, in this case it's interesting so I'll ask:

Does Rust do struct packing any differently than C and what kinds of tradeoffs are associated with that? Most people don't even think about struct packing in the context of C++ (which is in many ways closer to Rust) because of vtables / inheritance / etc, but in this case I'd like to know if Rust does anything differently or requires something new to think about in this respect.

Post reply on HN