Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Lost Art of Structure Packing (2018)

catb.org

Post reply on HN
Print view View original

31–40 of 120 posts

Re: The Lost Art of Structure Packing (2018)

#31

Post by shawxe »

banachtarski wrote: post #29
shawxe wrote: post #26

Earlier quoted context omitted.

Yeah, it's called memset. Not sure if this bug is supposed to be subtle or something, but if you require the padding bytes to be consistent then you need to consistently initialize your structs. Ignoring UB often leads to these sorts of bugs. EDIT: Based on some of the comments I'm getting here, it seems like some of you have never implemented a hashmap/generic interface in vanilla C and it shows. If you want a hashm…

Wrong answer. This is brittle and relies on the memory always being initialized correctly and will be prone to all sorts of issues in the wild. Better is just either templatize on the key type or store the size on the map.

To be clear, there are obviously different/better ways to handle this in C++. We're talking about K&R C here.

Re: The Lost Art of Structure Packing (2018)

#32

Post by shawxe »

saagarjha wrote: post #27
shawxe wrote: post #26

Earlier quoted context omitted.

Yeah, it's called memset. Not sure if this bug is supposed to be subtle or something, but if you require the padding bytes to be consistent then you need to consistently initialize your structs. Ignoring UB often leads to these sorts of bugs. EDIT: Based on some of the comments I'm getting here, it seems like some of you have never implemented a hashmap/generic interface in vanilla C and it shows. If you want a hashm…

I don’t think memset is required to zero out padding bytes. The correct way to do this is to only access non-padding bytes.

sizeof(struct foo) is required to return the actual size, including padding, so

struct foo foo_inst; memset(&foo_inst, 0, sizeof(foo_inst));

will certainly result in all padding bytes being set to zero.

Re: The Lost Art of Structure Packing (2018)

#35

Post by scatters »

shawxe wrote: post #26
kazinator wrote: post #20

Earlier quoted context omitted.

> C++20 now has a "hashable" concept to help with this I'm sorry, you can easily solve this problem in K&R C from 1979.

Yeah, it's called memset. Not sure if this bug is supposed to be subtle or something, but if you require the padding bytes to be consistent then you need to consistently initialize your structs. Ignoring UB often leads to these sorts of bugs. EDIT: Based on some of the comments I'm getting here, it seems like some of you have never implemented a hashmap/generic interface in vanilla C and it shows. If you want a hashm…

memset, sure, then you copy the struct (return it or pass it by value) and the compiler won't bother copying the padding bytes. Or worse, an optimizing compiler will see that you're writing to padding bytes and helpfully no-op it.

Re: The Lost Art of Structure Packing (2018)

#36

Post by kazinator »

shawxe wrote: post #26
kazinator wrote: post #20

Earlier quoted context omitted.

> C++20 now has a "hashable" concept to help with this I'm sorry, you can easily solve this problem in K&R C from 1979.

Yeah, it's called memset. Not sure if this bug is supposed to be subtle or something, but if you require the padding bytes to be consistent then you need to consistently initialize your structs. Ignoring UB often leads to these sorts of bugs. EDIT: Based on some of the comments I'm getting here, it seems like some of you have never implemented a hashmap/generic interface in vanilla C and it shows. If you want a hashm…

One solution consists of using memset to initialize the structure to zero, and using memcpy to copy it instead of structure assignment. (Problem: pass-by-value in function calls won't use memcpy; abstractly, it uses member-for-member assignment which is not required to copy padding. A function that wants to calculate the correct has has to prepare a blank object with memset, then individually assign the fields into it from the incoming object.)

Another solution, more along the lines of what I was thinking, is simply to associate the hash table with a hashing function which processes the type as a structure, hashing the members individually rather than as a pad of memory.

C++ templates refine this by adding the ability to deduce the hashing function statically, and possibly inline it, which we could do with some preprocessing in C, along the lines of how those TAILQ macros from BSD work for linked lists.

Re: The Lost Art of Structure Packing (2018)

#37

Post by WesolyKubeczek »

saagarjha wrote: post #28
throwaway_pdp09 wrote: post #23

It's probably not talked about so much now because of the canard that memory is cheap, and because HLLs disguise things a bit too much and so mislead newbies, but to suggest it's lost is plain wrong.

I think the main reason is that many new languages do this for you, at the cost of not guaranteeing a particular structure member order.

Sheesh, we're trying to be ABI-compatible over here.

Re: The Lost Art of Structure Packing (2018)

#38

Post by kazinator »

saagarjha wrote: post #27
shawxe wrote: post #26

Earlier quoted context omitted.

Yeah, it's called memset. Not sure if this bug is supposed to be subtle or something, but if you require the padding bytes to be consistent then you need to consistently initialize your structs. Ignoring UB often leads to these sorts of bugs. EDIT: Based on some of the comments I'm getting here, it seems like some of you have never implemented a hashmap/generic interface in vanilla C and it shows. If you want a hashm…

I don’t think memset is required to zero out padding bytes. The correct way to do this is to only access non-padding bytes.

memset is absolutely required to zero out padding bytes. Padding is part of the structure size, and memset hast to zero out exactly as many bytes as it is told.

memseting a structure to zero is commonly done in programs that send structure outside of the process (like passing it to communication or storage-related system calls) because the padding can leak sensitive information. That better work!

If the size of a structure didn't include the padding, then pointer arithmetic on structures wouldn't work correctly, and arrays of them would be broken/impossible. Arrays are the reason for the padding; given a struct foo * p, we need p + 1 to be properly aligned (for the sake of accessing all the members of * (p + 1). So struct foo cannot have a size like 5, if it contains a member of type int or anything else with alignment requirements.

Re: The Lost Art of Structure Packing (2018)

#39

Post by DagAgren »

WesolyKubeczek wrote: post #34
DagAgren wrote: post #33

It's 2020, do we really still need to pay attention to ESR?

It's been 5 minutes already, why pay attention to one DagAgren and their comments?

May I present my own achievements, such as: Not being an awful racist, and not spending any of my time defending pedophile rapists?

Re: The Lost Art of Structure Packing (2018)

#40

Post by kazinator »

scatters wrote: post #35
shawxe wrote: post #26

Earlier quoted context omitted.

Yeah, it's called memset. Not sure if this bug is supposed to be subtle or something, but if you require the padding bytes to be consistent then you need to consistently initialize your structs. Ignoring UB often leads to these sorts of bugs. EDIT: Based on some of the comments I'm getting here, it seems like some of you have never implemented a hashmap/generic interface in vanilla C and it shows. If you want a hashm…

memset, sure, then you copy the struct (return it or pass it by value) and the compiler won't bother copying the padding bytes. Or worse, an optimizing compiler will see that you're writing to padding bytes and helpfully no-op it.

A compiler that optimizes away memset is not one that could be used for targeting anything that sits on a network.
Post reply on HN