Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Structures in C: From Basics to Memory Alignment

abstractexpr.com

Post reply on HN
Print view View original

21–30 of 58 posts

Re: Structures in C: From Basics to Memory Alignment

#21

Post by owalt »

mananaysiempre wrote: post #9
com2kid wrote: post #6

Earlier quoted context omitted.

> All of these sound weird to me—most non-stupid (hello 802.2) protocols and hardware are going to have natural-aligned structure fields, so basically any mainstream (8-bit-byte, two’s complement, etc.) ABI is going to lay them out the same way, packed or not. In the long ago year of 2015 I worked on a project where the same binary packet was: 1. Generated by an 8 bit micro controller 2. Consumed by a 32bit Cortex M3…

> The phrase "natural aligned" has no meaning in that context. The phrase “naturally aligned” as I’m accustomed to seeing it used refers to the alignment of a power-of-two-sized type (usually a scalar one) being equal to its size. Unless you’re working with, say, 18-bit or 24-bit integers (that do exist in obscure places), it does have a meaning, and unless you’re using non-eight-bit bytes that meaning is fairly univ…

I do my current work (embedded) on an architecture with the following properties:

- 8-bit bytes

- 16-bit aligned accesses to 32-bit types

- 32-bit aligned accesses to 64-bit types.

- Struct alignment depends on the size of the struct (32-bit aligned for >= 64-bit structs)

It's a pretty common architecture in the automotive industry, though probably would be considered esoteric for other applications.

This is not the first platform I've encountered with "unnatural" alignment rules in the embedded space, and I'm sure it won't be the last. (The extra packing this allows is actually quite handy.)

Re: Structures in C: From Basics to Memory Alignment

#22

Post by synergy20 »

atahanacar wrote: post #20
synergy20 wrote: post #4

pretty cool, love anything related to C. might want to add anonymous struct. also put function pointers inside struct for simple object-oriented-programming in C. flexible array is handy, you do one malloc for all, but pointers inside struct is more 'flexible', for example you can put a 'void *' and cast it to various data types. for flexible array, the data types must be chosen first.

>but pointers inside struct is more 'flexible', for example you can put a 'void *' and cast it to various data types. But then the array and other members won't be next to each other in memory.

the cache subsystem might be smart enough to cache the first line, e.g. the size of the array etc, then pull the rest of array content from heap and cache the array as needed, should be fine to me, yes two cache misses instead of one at the beginning.

Re: Structures in C: From Basics to Memory Alignment

#24

Post by Dylan16807 »

colonwqbang wrote: post #3

> The only good reason to use packed structures is when you need to map some memory (e.g. hardware registers exposed to memory) bit by bit to a structure. Another common reason is when two CPUs of different architecture need to access the same structure in memory. E.g. you have a RiscV and an Arm64 processor in the same system, sharing memory. Or you read structured binary data from disk and need to specify an exact…

> Another common reason is when two CPUs of different architecture need to access the same structure in memory.

That sounds like mapping memory and something you could avoid with the right compiler flags.

> Or you read structured binary data from disk and need to specify an exact layout.

If you need it stay in the same form it has on the disk, I think it's fair to call that a concern in the category of "map some memory". Edit: And other spots in the text explicitly include file formats.

Basically I think you're giving examples rather than exceptions.

Re: Structures in C: From Basics to Memory Alignment

#25

Post by spacechild1 »

mananaysiempre wrote: post #9
com2kid wrote: post #6

Earlier quoted context omitted.

> All of these sound weird to me—most non-stupid (hello 802.2) protocols and hardware are going to have natural-aligned structure fields, so basically any mainstream (8-bit-byte, two’s complement, etc.) ABI is going to lay them out the same way, packed or not. In the long ago year of 2015 I worked on a project where the same binary packet was: 1. Generated by an 8 bit micro controller 2. Consumed by a 32bit Cortex M3…

> The phrase "natural aligned" has no meaning in that context. The phrase “naturally aligned” as I’m accustomed to seeing it used refers to the alignment of a power-of-two-sized type (usually a scalar one) being equal to its size. Unless you’re working with, say, 18-bit or 24-bit integers (that do exist in obscure places), it does have a meaning, and unless you’re using non-eight-bit bytes that meaning is fairly univ…

> the default ABI settings should get you completely compatible layouts here

That's not true! You must not assume that the alignment always equals the size of a type. For example, the SysV i386 ABI uses 32-bit alignment for 64-bit types (double, int64_t). The Microsoft x86 ABI, however, uses 64-bit alignment, as do all 64-bit ABIs (See https://stackoverflow.com/a/11110283.) If you want to share structs directly between different machines, you should use appropriate struct packing directives - unless you really know what you are doing.

Re: Structures in C: From Basics to Memory Alignment

#26

Post by wahern »

colonwqbang wrote: post #14
mananaysiempre wrote: post #9

Earlier quoted context omitted.

> The phrase "natural aligned" has no meaning in that context. The phrase “naturally aligned” as I’m accustomed to seeing it used refers to the alignment of a power-of-two-sized type (usually a scalar one) being equal to its size. Unless you’re working with, say, 18-bit or 24-bit integers (that do exist in obscure places), it does have a meaning, and unless you’re using non-eight-bit bytes that meaning is fairly univ…

I think we agree that this makes sense on some metaphysical level. The problem is that there are definitely platforms where the normal alignment isn't what you describe above. And there isn't to my knowledge a switch in GCC to force it to follow these rules on any given platform. There isn't __attribute__((natural_alignment)). But there is __attribute__((packed)).

Since C11 there is _Alignas(sizeof T), forcing one of the proposed meanings for alignment, and _Alignof(T), which queries actual (i.e. natural, per another meaning) alignment. But, yeah, the argument upthread seems more about the implicit meaning of natural than anything else.

Re: Structures in C: From Basics to Memory Alignment

#27

Post by kazinator »

somenits wrote: post #18

Good article, a couple nits/additional notes: 1. The article points out you should compare structs field by field, but it doesn't explain why memcmp wouldn't work. The reason is that the padding between the fields might not necessarily be zeroed in all cases. Field by field comparison is resilient to this. 2. The article proposes this for dynamic allocation: struct Vector2D *vec = malloc(sizeof(struct Vector2D)); I t…

But if you change the type name, you are probably also doing a search and replace.

In the end, the only justification for the latter rather than the former is that the pointer name is often a short name like vec, and so sizeof *vec is shorter than the alternative.

That has advantages.

If there is a typo in a short name, you're more likely to see it.

I can easily miss the typo:

  struct Vector3D *vec = malloc(sizeof(struct Vector2D));
The sizeof also doesn't require parentheses, so it's shorter by two characters right off the bat.

Re: Structures in C: From Basics to Memory Alignment

#28

Post by throwaway894345 »

> The great thing about [passing structs by copy] is that it allows us to omit to allocate structs with malloc on the heap. Instead, we could create a struct on the stack and pass it around by copying it. This opens the door to new approaches for writing safer C code.

You don’t have to heap-allocate structs to pass by pointer; you can pass a pointer to a struct living on the stack. Even if you are talking about returning a struct, you can just take a pointer to memory allocated in some previous stack frame and mutate it—this avoids the allocation and thus preserves safety and performance. But yes, being able to pass structs by copy is nice too.

Re: Structures in C: From Basics to Memory Alignment

#29

Post by throwaway894345 »

somenits wrote: post #18

Good article, a couple nits/additional notes: 1. The article points out you should compare structs field by field, but it doesn't explain why memcmp wouldn't work. The reason is that the padding between the fields might not necessarily be zeroed in all cases. Field by field comparison is resilient to this. 2. The article proposes this for dynamic allocation: struct Vector2D *vec = malloc(sizeof(struct Vector2D)); I t…

I’ve been wondering about (1) recently—is there a way to memset the entire stack frame at the start of a function such that memcmp works as expected? Also, what are the performance implications of comparing a padded struct member-by-member vs a single big memcmp? Is member-by-member faster because you’re comparing less in total, or is memcmp faster because it’s one big contiguous compare? Or is it more complicated?

Regarding (2), sizeof(*x) doesn’t actually dereference x, right? The dereference isn’t evaluated—it’s all calculated at compile time, right?

Re: Structures in C: From Basics to Memory Alignment

#30

Post by kazinator »

colonwqbang wrote: post #3

> The only good reason to use packed structures is when you need to map some memory (e.g. hardware registers exposed to memory) bit by bit to a structure. Another common reason is when two CPUs of different architecture need to access the same structure in memory. E.g. you have a RiscV and an Arm64 processor in the same system, sharing memory. Or you read structured binary data from disk and need to specify an exact…

Unless you're using weird C compilers, you shouldn't need to pack structures in order to have RiscV and Arm64 access the same structure. Packing structures will be detrimental in a situation where any of the processors have alignment requirements for accessing words. Extra instructions may be required to access that four byte member at offset 13, by using two accesses and some shifting and masking. Atomicity has gone out the window.

If we look at, say, GCC, the structure layout rules are very consistent across the targets. They are not absolutely the same, but consistent enough to work with.

Packing won't take care of byte order, though it's not super common to have a system where hosts of opposite byte order are accessing the same buses, and sharing memory. (I've worked on systems where a DPS had "weird order", like opposite endian from the host, but in 16 bit units. Not DCBA but CDBA or something like that.)

You may be tempted to use a packed structure for conforming to some layout in a network packet or on disk, but it leaves the code nonportable and slow. In between the time the packed structure is read from the file and written again, it might be accessed many times. All those times may be slowed down due to the packing. You may be better off writing proper serializing and deserializing. That can also help if there are multiple versions of the format, or some optional fields and other crap. You may be able to map all the variants onto a single in-memory structure, dealing with the differences only in the serializing and deserializing routines.

I'm skeptical of how common it is to need packing in order to map hardware registers bit-by-bit. Hardware registers tend to be very regularly sized and spaced. If you're literally mapping bit-by-bit, the language extension that is most beneficial is the ability to declare a bitfield to be of any integer type. E.g. if registers are consecutive 32 bit words:

   // Register AREG (32 bits)
   uint32_t AREG_foo_field : 13;
   uint32_t AREG_bar_bit : 1;

   uint32_t : 0;  // done with this cell: go to next

   // Register BREG
   uint32_t BREG_bozo_bit : 1;
Bitfield allocation is endian-dependent. In the end, for utmost portability, you want to access the 32 bit word as a word, and do the shifting and masking, where you can add a byte swap, if necessary.
Post reply on HN