141–150 of 277 posts
Earlier quoted context omitted.
Are you accepting bug reports for such issues in tarsnap? :)
Yes. If you look at the commit history you'll find that we've fixed a few of them already.
Earlier quoted context omitted.
That's the protobuf approach, but things like Capnp are different, right? Isn't its wire format its in-memory format? I thought it was closer to the "straight memcpy" approach. Protobuf serialization functions can turn into a lot of generated code too (read: big binaries) and stomp all over your icache, because they're different for each type. More "data-driven" approaches using type-generic functions with some runti…
There are dozens, if not hundreds of serialization libraries. None of them are going to compare to something like sending a raw struct on the wire, if you need absolute performance. Probably the biggest advantage of things like grpc, flatbuffers, capnproto, etc, are they provide serialization libraries for many different languages. Sending a raw struct will be up to the implementer to get the other side correct. And…
> None of them are going to compare to something like sending a raw struct on the wire, if you need absolute performance.
I don't know about that.
For simple flat structs containing integers only (no pointers), Cap'n Proto is nearly identical to raw structs.
For complex structures with pointers, sending raw structs doesn't work.
Admittedly you may suffer unwanted code bloat linking in the Cap'n Proto library if all you really want to do is send a flat, raw struct.
> And by sticking with the serialization libraries with higher performance, like flatbuffers and capnproto, you also lose a little bit of future flexibility, since its typically harder to change the format of these structures compared to higher level libraries.
I disagree with this. Cap'n Proto's compatibility story is essentially the same as Protobuf and JSON: You can add new fields and old programs will ignore them.
(I suspect Flatbuffers is also similar but I haven't looked at it in a long time.)
Earlier quoted context omitted.
Surely you only need memset_s if you're doing this for security reasons, as if memset is optimized away it's because the compiler can prove the observable semantics are the same.
Nope, memset will often be “optimized” such that padding is not zeroed. It’s not because the compiler sees that padding isn’t observed, it’s because reading from padding is undefined. Because reading from padding is undefined the write (also undefined) by definition has no observable side effects. Memset_s is the only way to guarantee that padding is actually zeroed, well memset or bzero_s obviously. It’s critically…
[1] says that memset_s exists because it is not optimized away if it just clears a variable that will not be used after the call (unlike memset)
On the other end: people who do this for security reasons say not to trust memset_s is 'an optional feature of c11 and not really portable' [2]
[1] https://en.cppreference.com/w/c/string/byte/memset
[2] https://www.cryptologie.net/article/419/zeroing-memory-compi...
Earlier quoted context omitted.
Packed structs are reasonably standard (even gcc and visual studio can be made to pack in the same way using the same syntax these days). The remaining issue is endianness, but you can configure ARM to match x86 at boot so shrug . Also, if I tell any reasonably experienced developer my protocol sends a packed struct with these fields: uint8_t version uint16_t reserved uint8_t op int32_t status uint32_t data_length //…
My experience with packed structs is there really aren't any gotcha's. Either works or not. The big annoyance is network people all use big endian probably just to spite everyone else.
Earlier quoted context omitted.
Sending structures that include padding or that you don’t know the exact layout of over the network is another thing you shouldn’t do, period. Code doing that may break with a compiler change or a compilation flag change on one end of the connection, even if you can guarantee both ends use the same CPU.
There is an attribute that lets you specify alignment per-struct, which means you will know the exact layout.
Earlier quoted context omitted.
Use a packed struct (e.g. __attribute__((packed))). You may take a performance hit due to lack of alignment, but that's the judgement call
> performance hit due to lack of alignment This isn't true of modern processors.
Earlier quoted context omitted.
You're probably not serious, but why waste time and complexity copying structure elements into a buffer (which likely needs to be memset/bzero'd!), if you can just directly use the struct, all consistent with the standard? C is not a very high-level language. Apart from where undefined behavior is called out, you can and should assume how data is represented on a low level. It was made for writing operating systems a…
I'm joking because endless flame wars have already been spent debating this issue. But I realize not everyone has heard them yet, so let's strap on the football shoes and bring out the horse carcass one more time! There are platforms on which NULL, integer 0 and floating point 0 is not the same. Hence memsetting won't do the right thing. On modern platforms they are, but still, you don't want to get the Standard C We…
Am I the only one who finds this int yes=1; setsockopt(listener, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof (yes)) ; much cleaner than this setsockopt(listener, SOL_SOCKET, SO_REUSEADDR, &(int) {1}, sizeof(int)); ?