Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Packed structs in Zig make bit/flag sets trivial

devlog.hexops.com

Post reply on HN
Print view View original

81–90 of 147 posts

Re: Packed structs in Zig make bit/flag sets trivial

#81

Post by pclmulqdq »

stephencanon wrote: post #77
Joker_vD wrote: post #51

Earlier quoted context omitted.

Uh, that's standard C89. You know, the bit fields? Except for the "false" keyword, that's from stdbool.h and you need C99 for that to reliably exist.

The only real problem with C (and C++) bitfields is that the standard doesn’t say anything about how they’re laid out in memory, which unfortunately makes them mostly useless unless you either don’t care about layout or you’re targeting one specific compiler and that compiler documents its choices and doesn’t break them in a few years. Sign-extension of 1-bit fields also messes people up all the time, but that’s “jus…

That is a really big problem in some applications. I wish the C committee would come out and say "use the layout that GCC 11 .x.x does" to make these a lot more usable.

Re: Packed structs in Zig make bit/flag sets trivial

#82

Post by pclmulqdq »

scaredginger wrote: post #68
flumpcakes wrote: post #41

Earlier quoted context omitted.

Wow... I thought I had seen a lot of strange C++ code but I have _never_ seen packed structs using this syntax: struct X { type alias : numeric_value = false; ... } I love the power of C++ but there is _so much_ to the language. I'm sure there would also be some template meta programming solution even if this syntax was available.

It's probably tough to find a metaprogramming solution to this, for two reasons: 1. Addressing happens at the byte level, not the bit level, so a type can't begin on any bit. You'd have to do your own addressing, such as in std::bitset. 2. For now, there's no reflection in the language, so you can't really assign names to members in a general way (hello, preprocessor). A solution might be to index by type; something…

I have done similar things a few times, and like JSON handling, the optimal API does not seem to be the same for every intended use. Size-focused cases want different things than speed-focused ones, and you need to deal with sign extension, etc.

Re: Packed structs in Zig make bit/flag sets trivial

#83

Post by scaredginger »

galangalalgol wrote: post #79
scaredginger wrote: post #50

Earlier quoted context omitted.

The in-memory representation of bit fields is implementation-defined. Therefore, if you're calling into an external API that takes a uint32_t like in the example without an explicit remapping, you may or may not like the results. In practice, everything you're likely to come across will be little endian nowadays, and the ABI you're using will most likely order your struct from top to bottom in memory, so they will lo…

I've dealt with oddities talking between big endian powerpc using these. Its been a few years but the difference wasn't just the endianess I think? Still, dealing with the mapping was way easier than masking for large structs. Is big endian really dead now?

As I said, it's also ABI. Though admittedly, endianness would be encompassed by ABI, so it's all really just ABI

Re: Packed structs in Zig make bit/flag sets trivial

#84

Post by lsof »

masklinn wrote: post #76
lsof wrote: post #52

Earlier quoted context omitted.

Red is the least significant bit, because offsets start at bit #0. Endianness is not a concern because you have to specify the underlying integer type, and bit endianness is not a thing, so it's just the native endianness (if you, e.g., need a network format with specific endianness and a 32-bit bitfield, you can instead use 4 consecutive 8-bit bitfields laid out in a consistent way). Unspecified bits are ignored; th…

> Red is the least significant bit, because offsets start at bit #0. So red is the LSB because you decided it was the LSB. That is not a by-definition thing. > Endianness is not a concern because you have to specify the underlying integer type, and bit endianness is not a thing That’s not actually true. There are formats which process bytes LSB to MSB, and formats which process them MSB to LSB. E.g. git’s offset-enco…

> I’m quite confused by “bitfield” do you mean the container field (the one that’s actually defined by the `bitfield` keyword) or the contained sub-fields?

By 'field' I meant the contained sub-fields

Re: Packed structs in Zig make bit/flag sets trivial

#85

Post by chrismorgan »

magicalhippo wrote: post #71
chrismorgan wrote: post #58

Earlier quoted context omitted.

I’ve definitely desired an operator that combines these before. I’ve settled on &== as the best syntax in C-style languages, with a &== b being equivalent to a & b == b (but with a single evaluation of b). if (mask &== WGPUColorWriteMask_Alpha|WGPUColorWriteMask_Blue) { … } I find only two very minor problems with it: firstly, that it’s not commutative (that is, a &== b is not equivalent to b &== a ). Secondly, that…

> I find only two very minor problems with it: firstly, that it’s not commutative Why is that a problem? That is, if a has additional bits set compared to b, then ((a & b) == b) != ((b & a) == a), no?

Of common operators, / and - are the only ones that aren’t commutative.

For equality comparisons, there are two opposing conventions: actual == expected (the more popular, in my experience), and expected == actual (by no means rare). Having &== and |== be order-sensitive (though there’s absolutely no question in my mind about what the ordering should be) is mildly unfortunate.

It’s very minor.

Re: Packed structs in Zig make bit/flag sets trivial

#86

Post by cesaref »

shultays wrote: post #53
sqrt_1 wrote: post #22

Earlier quoted context omitted.

Here is a sample of how you do this in C++ https://godbolt.org/z/W3bMcqM5P

I thought you were not able to initialize bitfield members like that in struct. Am I misremembering, was that something else or is it compiler/c++ release dependent?

This is a c++20 feature

Re: Packed structs in Zig make bit/flag sets trivial

#87

Post by MontyCarloHall »

scaredginger wrote: post #80
MontyCarloHall wrote: post #73

Earlier quoted context omitted.

This has been possible since C89. See 3.5.2.1 of the C89 rationale [0]. It is architecture/compiler dependent though. This is explicitly acknowledged in the rationale document: “Since some existing implementations, in the interest of enhanced access time, leave internal holes larger than absolutely necessary, it is not clear that a portable deterministic method can be given for traversing a structure field by field.”…

I think they were referring to the default member initializer, a feature added in C++11 not exclusive to bitfields. I couldn't see any reference to anything similar in what you linked (admittedly, I skimmed though). For example: struct X { int defaulted = 1; // 1 is the default member initializer };

>I think they were referring to the default member initializer

Ah, that would make more sense. The combination of bitfield and default initializer syntax does look especially odd, since both features are rarely used IME.

>I couldn't see any reference to anything similar in what you linked

Indeed, there is no way to specify default values for a C struct at definition time.

Re: Packed structs in Zig make bit/flag sets trivial

#88

Post by mhh__ »

eqvinox wrote: post #59
tempodox wrote: post #31

Earlier quoted context omitted.

Bit fields in C are notoriously non-portable (exact physical layout depends on the compiler). Is Zig any better (apart from the fact that there is only one Zig compiler for now)?

Notoriously? I can't name any architecture/platform/compiler where bits aren't allocated consecutively in order of definition, without holes, in the same direction as system endianness...

gcc?

MSVC does things simply, gcc does not

Re: Packed structs in Zig make bit/flag sets trivial

#89

Post by mhh__ »

WalterBright wrote: post #3
MobiusHorizons wrote: post #2

This is a nice syntax, but the functionality seems essentially the same as bit fields in C. Does this provide anything additional?

D just went ahead and implemented bit fields. They work, and it's hard to find any way to improve on them.

It's hard but they aren't perfect. You shouldn't rely on them for packing memory at all if you want to talk to other programs.

Similarly bitfields mean you end up with "ints" that are actually 3 bits wide and so on.

Re: Packed structs in Zig make bit/flag sets trivial

#90

Post by vitus »

fanf2 wrote: post #65
eqvinox wrote: post #62

Earlier quoted context omitted.

> It is not possible to specify bitfields in a way that makes sense, is consistent with a bytewise view of memory, and is portable between big endian and little endian processors. I'm not sure I accept "consistency with a bytewise view of memory" as a well-defined, reasonable concept. I do expect to give a list of bit widths, and get a field that has these in consecutive order. Why would it randomly do weird things o…

> > If the fields are allocated in order, is u4 the low bits of the first byte or the high bits? > It's the low bits on LE, and the high bits on BE. You are advocating for the current rule in C. As I said, C's rule implies 8 bit fields will be in different orders in memory on machines of different endianness, which makes it very difficult to use bitfields to get exact control over memory layout in a portable manner.

Is that not why functions like htonl exist: to convert between the host architecture's endianness and a platform-independent representation?

Your binary isn't going to be portable. The only reason your memory layout should be is if you intend to serialize it. But if you're going that extra step, you _need_ to convert it to a platform-independent format regardless -- otherwise, not even your ints deserialize correctly.

Post reply on HN