Live data from Hacker News

Intrusive linked lists (2019)

data-structures-in-practice.com

41–50 of 80 posts

Re: Intrusive linked lists (2019)

#41

Earlier quoted context omitted.

> If an application uses two libraries, each of which handles lists of int, each library will have to independently macro-generate its separate list implementation, and because C's type system is nominal, the generated types will be isomorphic but incompatible. Um, what? C89, 3.1.2.6: "Moreover, two structure, union, or enumeration types declared in separate translation units are compatible if they have the same numb…

> as long as the struct tags are the same Exactly. Now you have a naming problem. You need a naming convention that every user of the list library must follow, or else their types will be incompatible. And what about typedefs? If A is a typedef of B, or more generally A and B are typedef-related (their normal forms, obtained by following all typedefs, are the same), lists of A and B will be incompatible unless users…

> Now you have a naming problem. You need a naming convention that every user of the list library must follow, or else their types will be incompatible.

Oh, that's simple: just have empty struct tags.

> And what about typedefs?

The names introduced by the typedefs are irrelevant.

> A and B are typedef-related (their normal forms, obtained by following all typedefs, are the same), lists of A and B will be incompatible unless users agree on a common name.

Huh?

    typedef struct { int x; } A;
    typedef struct { int x; } B;

    typedef struct { header_list header; A payload; } list_of_A;
    typedef struct { header_list header; B payload; } list_of_B;
The structs list_of_A and list_of_B are compatible.

Re: Intrusive linked lists (2019)

#42

Earlier quoted context omitted.

> There is no OoB access of an array Yes, there is. It does not matter that storage happens to be allocated beyond the end of said array. Strict aliasing implies that it is UB to reinterpret the array as anything else. And it is UB to access an array out of bounds. Flexible array members specifically exist for these dynamically-allocated trailing arrays. They do not solve the strict aliasing problem, though. > if the…

> It does not matter that storage happens to be allocated beyond the end of said array. It does matter, for malloc-returned storage. You can put whatever objects you want into that storage as long as it fits and the pointer is properly aligned. > Strict aliasing ...is not violated; memcpy takes a void pointer as its destination, sets the effective type of the storage behind it, and the treats it as an array of unsign…

> It does matter, for malloc-returned storage. You can put whatever objects you want into that storage as long as it fits and the pointer is properly aligned.

You can certainly store an object of arbitrary type, but here it is done through a pointer to an object with pointer arithmetic going beyond the allowed bounds.

> memcpy takes a void pointer as its destination, sets the effective type of the storage behind it

And, in doing so, may very well overwrite the unspecified padding following 'payload' in the structure, thus instantly destroying the effective type of the structure object itself. Subsequent accesses to the structure or its members will be UB.

It seems to me that your argument hinges on two assumptions:

    - there is no padding following 'payload' (this would have to be statically asserted),
    - the pointer to 'payload' is indistinguishable from the pointer past the structure; in particular, provenance is not an issue.
That is a very interesting discussion.

Re: Intrusive linked lists (2019)

#43
post #10

Earlier quoted context omitted.

You have it backwards, an intrusive linked list is a linked list that is embedded in another data structure. The classic example is a linked list whose elements live on the stack. The article is wrong too, or at least using the term over-specifically. It's not really tied to C++isms at all.

GP points out that what the article calls "intrusive linked list" is a regular linked list . Wikipedia for instance gives the canonical linked list example of a struct with one embedded integer and a next link and of course does not call it "intrusive linked list". "Intrusive" got popular with C++ intrusive pointers, and that is where the article gets is misinformation from. And of coursed the web jockeys downvote th…

>"Intrusive" got popular with C++ intrusive pointers

It got popular with C++'s attempts at type safety. In particular, std::list lets you accomplish the machinery without macros, and allowing for polymorphism (heterogeneous lists of derived instances) without weird type casts and overallocation tricks, but at the cost of another level of indirection.

Re: Intrusive linked lists (2019)

#44

Earlier quoted context omitted.

Zero-sized arrays are not standard. Accessing an array out of bounds is UB. At the very least, you should use a flexible array member instead (char payload[];). But even if you did that, strict aliasing implies that 'payload' can only be accessed as an array of character type. It is correct to memcpy between 'payload' and another object of arbitrary type T of the appropriate size (as list_push_ does in your example),…

In practice you wouldn’t have the payload in the struct at all, just a fixed offset aligned with the maximum alignment, but this is more illustrative of what’s happening for an example. IIRC GCC and Clang lets character types alias to any type. Otherwise glibc’s malloc also doesn’t abide to strict aliasing.

> IIRC GCC and Clang lets character types alias to any type.

It is always legal to access the memory representation of any object as an array of characters. The other way around (interpreting an array of characters as a T, even though it does not have effective type T) is not.

> Otherwise glibc’s malloc also doesn’t abide to strict aliasing.

It may not have to. From the point of view of C, malloc is special because it is part of the implementation. The compiler is free to handle UB as it sees fit. In particular, it can decide that aliasing has different semantics in malloc.c than outside it.

Re: Intrusive linked lists (2019)

#45

I was surprised to see the main benefit of intrusive linking mentioned as a bit of a side note: The ability to move data around between lists (and within a list) without copying. You also get O(1) removal from the middle of the list, assuming you have a pointer to the object somewhere else. As a result, when you have large state structs and you don't do a lot of list scans, intrusive linking makes things a lot faster…

I'm pretty sure the author there means to compare the intrusive linked list to a non-intrusive linked list (such as C++ std::list), not to vectors etc. that aren't "linked" at all. As described e.g. in https://news.ycombinator.com/item?id=49549542 .

Re: Intrusive linked lists (2019)

#46

Earlier quoted context omitted.

> as long as the struct tags are the same Exactly. Now you have a naming problem. You need a naming convention that every user of the list library must follow, or else their types will be incompatible. And what about typedefs? If A is a typedef of B, or more generally A and B are typedef-related (their normal forms, obtained by following all typedefs, are the same), lists of A and B will be incompatible unless users…

> Now you have a naming problem. You need a naming convention that every user of the list library must follow, or else their types will be incompatible. Oh, that's simple: just have empty struct tags. > And what about typedefs? The names introduced by the typedefs are irrelevant. > A and B are typedef-related (their normal forms, obtained by following all typedefs, are the same), lists of A and B will be incompatible…

No, any tagless type is unique, so neither A and B nor list_of_A and list_of_B are compatible.

This is what I like to fix in C2y outside of typedefs (and it would really help if you file wishlist bugs with compilers if you agree).

Re: Intrusive linked lists (2019)

#48

Earlier quoted context omitted.

> as long as the struct tags are the same Exactly. Now you have a naming problem. You need a naming convention that every user of the list library must follow, or else their types will be incompatible. And what about typedefs? If A is a typedef of B, or more generally A and B are typedef-related (their normal forms, obtained by following all typedefs, are the same), lists of A and B will be incompatible unless users…

> Now you have a naming problem. You need a naming convention that every user of the list library must follow, or else their types will be incompatible. Oh, that's simple: just have empty struct tags. > And what about typedefs? The names introduced by the typedefs are irrelevant. > A and B are typedef-related (their normal forms, obtained by following all typedefs, are the same), lists of A and B will be incompatible…

> The structs list_of_A and list_of_B are compatible.

No, they are not. From C23, 6.7.3.4 Tags: Each declaration of a structure, union, or enumerated type which does not include a tag declares a distinct type.

Re: Intrusive linked lists (2019)

#50

Earlier quoted context omitted.

> It does not matter that storage happens to be allocated beyond the end of said array. It does matter, for malloc-returned storage. You can put whatever objects you want into that storage as long as it fits and the pointer is properly aligned. > Strict aliasing ...is not violated; memcpy takes a void pointer as its destination, sets the effective type of the storage behind it, and the treats it as an array of unsign…

> It does matter, for malloc-returned storage. You can put whatever objects you want into that storage as long as it fits and the pointer is properly aligned. You can certainly store an object of arbitrary type, but here it is done through a pointer to an object with pointer arithmetic going beyond the allowed bounds. > memcpy takes a void pointer as its destination, sets the effective type of the storage behind it A…

If payload was ever dereferenced as a char array as well, I would buy the strict aliasing argument. But it’s not, it exists as a char pointer solely for pointer arithmetic.

AFAIK The purpose of strict aliasing rules is to let the compiler assume that dereferencing pointers of different types never refer to the same memory.

If ISO C treats this as UB, shouldn’t ISO C be fixed?

Post reply on HN