Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Packed structs in Zig make bit/flag sets trivial

devlog.hexops.com

Post reply on HN
Print view View original

71–80 of 147 posts

Re: Packed structs in Zig make bit/flag sets trivial

#71

Post by magicalhippo »

chrismorgan wrote: post #58
flohofwoe wrote: post #18

Earlier quoted context omitted.

Yes it would need to be: if ((mask & (WGPUColorWriteMask_Alpha|WGPUColorWriteMask_Blue)) == (WGPUColorWriteMask_Alpha|WGPUColorWriteMask_Blue)) { ... } ...which is quite a mouthful.

I’ve definitely desired an operator that combines these before. I’ve settled on &== as the best syntax in C-style languages, with a &== b being equivalent to a & b == b (but with a single evaluation of b). if (mask &== WGPUColorWriteMask_Alpha|WGPUColorWriteMask_Blue) { … } I find only two very minor problems with it: firstly, that it’s not commutative (that is, a &== b is not equivalent to b &== a ). Secondly, that…

> I find only two very minor problems with it: firstly, that it’s not commutative

Why is that a problem? That is, if a has additional bits set compared to b, then ((a & b) == b) != ((b & a) == a), no?

Re: Packed structs in Zig make bit/flag sets trivial

#72

Post by tempodox »

fanf2 wrote: post #48
yellowapple wrote: post #35

Earlier quoted context omitted.

That's the point of a packed struct: to explicitly declare the physical layout (instead of letting the compiler optimize it). If a packed struct ends up being non-portable between Zig implementations, then my impression is that one and/or the other must have a bug that must be fixed in order to actually comply with the language spec.

It is not possible to specify bitfields in a way that makes sense, is consistent with a bytewise view of memory, and is portable between big endian and little endian processors. Say you have, for instance (using C notation) struct { unsigned one : 8; unsigned two : 8; }; The fields are supposed to be represented in memory in the same order they are declared, so one is the first byte and two is the second byte. This s…

Variations due to endianness (hardware platform) are to be expected but variations due to compiler (implementation-defined) can be avoided if the spec says so. The fact that so many CPUs are little-endian these days certainly doesn't make things easier.

Re: Packed structs in Zig make bit/flag sets trivial

#73

Post by MontyCarloHall »

scaredginger wrote: post #66
eqvinox wrote: post #57

Earlier quoted context omitted.

You can absolutely initialize bitfield members in structs, not sure where your memory is from… maybe it was broken on some specific compiler vendors/versions?

Possible they were just pre-C++11

This has been possible since C89. See 3.5.2.1 of the C89 rationale [0].

It is architecture/compiler dependent though. This is explicitly acknowledged in the rationale document:

“Since some existing implementations, in the interest of enhanced access time, leave internal holes larger than absolutely necessary, it is not clear that a portable deterministic method can be given for traversing a structure field by field.”

[0] https://www.lysator.liu.se/c/rat/c5.html

Re: Packed structs in Zig make bit/flag sets trivial

#74

Post by masklinn »

zppln wrote: post #47
masklinn wrote: post #45

Earlier quoted context omitted.

Yes? If you have lots of bitfields and the convenience / readability is worth it, why wouldn’t you?

I meant as opposed to having it built in.

There are lots of reasons to not want it built-in e.g. it makes the language more complex, and if bad semantics are standardised you’re stuck with them.

If the language supports implementing a feature externally then it’s a good thing, as it allows getting wide experience with the feature without saddling the language with it, and if the semantics are fine and it’s in wide-spread use, then nothing precludes adding it to the language later on.

It’s much easier to add a feature to a language than to remove it.

Re: Packed structs in Zig make bit/flag sets trivial

#75

Post by layer8 »

shultays wrote: post #54
jeffreygoesto wrote: post #38

Earlier quoted context omitted.

The order of a bitfield is explicitly implementation defined, at least in C++ [0] and C99. I have been bitten by this when compiling bit-fiddling code with a different compiler (for the same platform). [0] section "Notes" in https://en.cppreference.com/w/cpp/language/bit_field

Interesting, I saw some low level code that assumes that bits are neatly packed. Some C decisions really confuses me. What would be the point of this one?

The alignment restrictions for memory accesses on some architectures may prevent bitfields from crossing certain address boundaries, hence implementations may need to reorder and/or pad the fields. The alternative would have been to either fail compilation for bitfield combinations that are incompatible with the target platform, which would arguably be worse, because you couldn’t portably use bitfields at all then, or to force implementations to generate code with case distinctions, possibly depending on the actual alignment at runtime, performing the bitfield accesses using multiple memory accesses if necessary.

Re: Packed structs in Zig make bit/flag sets trivial

#76

Post by masklinn »

lsof wrote: post #52
masklinn wrote: post #42

Earlier quoted context omitted.

The one thing that would worry me here is the endianness e.g. is red the LSB or the MSB? Also what happens to the padding (bits not covered by subfields)? How’s that going to look when shoved into a file or over a socket? how does the langage handle overflow (more bits in the bit fields than there are in the parent field)?

Red is the least significant bit, because offsets start at bit #0. Endianness is not a concern because you have to specify the underlying integer type, and bit endianness is not a thing, so it's just the native endianness (if you, e.g., need a network format with specific endianness and a 32-bit bitfield, you can instead use 4 consecutive 8-bit bitfields laid out in a consistent way). Unspecified bits are ignored; th…

> Red is the least significant bit, because offsets start at bit #0.

So red is the LSB because you decided it was the LSB. That is not a by-definition thing.

> Endianness is not a concern because you have to specify the underlying integer type, and bit endianness is not a thing

That’s not actually true. There are formats which process bytes LSB to MSB, and formats which process them MSB to LSB. E.g. git’s offset-encoding, the leading byte is a bitmap of continuation bytes, bit 0 indicates whether byte 7 is present.

Both are perfectly justifiable, one is offset-based, while the other is visualisation-based as bytes are usually represented MSB-first (as binary numbers, essentially).

> but when reading a field only its specified bits are read so unspecified bits doesn't change that result. When writing to a field, the source value is truncated to the field size, so you never end up writing to other bits.

I’m quite confused by “bitfield” do you mean the container field (the one that’s actually defined by the `bitfield` keyword) or the contained sub-fields?

Re: Packed structs in Zig make bit/flag sets trivial

#77

Post by stephencanon »

Joker_vD wrote: post #51
flumpcakes wrote: post #41

Earlier quoted context omitted.

Wow... I thought I had seen a lot of strange C++ code but I have _never_ seen packed structs using this syntax: struct X { type alias : numeric_value = false; ... } I love the power of C++ but there is _so much_ to the language. I'm sure there would also be some template meta programming solution even if this syntax was available.

Uh, that's standard C89. You know, the bit fields? Except for the "false" keyword, that's from stdbool.h and you need C99 for that to reliably exist.

The only real problem with C (and C++) bitfields is that the standard doesn’t say anything about how they’re laid out in memory, which unfortunately makes them mostly useless unless you either don’t care about layout or you’re targeting one specific compiler and that compiler documents its choices and doesn’t break them in a few years.

Sign-extension of 1-bit fields also messes people up all the time, but that’s “just” an easy-to-fix bug.

Re: Packed structs in Zig make bit/flag sets trivial

#78

Post by Comevius »

flohofwoe wrote: post #16

One area where this is still less flexible than regular bit masking is that quite often I want to access groups of bits as a value. For instance if the bits represent in/out pins on a chip emulator, sometimes I want to access unique data bus pins, and sometimes I want to set or get the data bus value. Maybe Zig can handle this case with unions though, I haven't tried this yet (this would require that unions can work…

You can do this and more in Zig. The only caveat is that for MMIO you need a volatile pointer.

https://www.scattered-thoughts.net/writing/mmio-in-zig/

It gets tricky if you need more control over loads and stores, or if there are different address spaces.

Re: Packed structs in Zig make bit/flag sets trivial

#79

Post by galangalalgol »

scaredginger wrote: post #50
MobiusHorizons wrote: post #2

This is a nice syntax, but the functionality seems essentially the same as bit fields in C. Does this provide anything additional?

The in-memory representation of bit fields is implementation-defined. Therefore, if you're calling into an external API that takes a uint32_t like in the example without an explicit remapping, you may or may not like the results. In practice, everything you're likely to come across will be little endian nowadays, and the ABI you're using will most likely order your struct from top to bottom in memory, so they will lo…

I've dealt with oddities talking between big endian powerpc using these. Its been a few years but the difference wasn't just the endianess I think? Still, dealing with the mapping was way easier than masking for large structs. Is big endian really dead now?

Re: Packed structs in Zig make bit/flag sets trivial

#80

Post by scaredginger »

MontyCarloHall wrote: post #73
scaredginger wrote: post #66

Earlier quoted context omitted.

Possible they were just pre-C++11

This has been possible since C89. See 3.5.2.1 of the C89 rationale [0]. It is architecture/compiler dependent though. This is explicitly acknowledged in the rationale document: “Since some existing implementations, in the interest of enhanced access time, leave internal holes larger than absolutely necessary, it is not clear that a portable deterministic method can be given for traversing a structure field by field.”…

I think they were referring to the default member initializer, a feature added in C++11 not exclusive to bitfields. I couldn't see any reference to anything similar in what you linked (admittedly, I skimmed though).

For example:

    struct X {
        int defaulted = 1; // 1 is the default member initializer
    };
Post reply on HN