Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BuffDB is a Rust library to simplify multi-plexing on edge devices

github.com

Post reply on HN
Print view View original

11–20 of 33 posts

Re: BuffDB is a Rust library to simplify multi-plexing on edge devices

#11

Post by 38 »

ithkuil wrote: post #10
38 wrote: post #9

Earlier quoted context omitted.

> protobuf such an awful format, I wish people would stop using it

Why? The format is not that bad. The binding/libraries OTOH are often awful and they often require unnecessary full-message deserialization

not self describing. if it was just the field names I could deal with that, but even the values are ambiguous, since the same type is used for Bytes and embedded Messages. the worst part is the wire type integer has two unused values, so they easily could have added a wire type for embedded messages

Re: BuffDB is a Rust library to simplify multi-plexing on edge devices

#12

Post by cryptonector »

ithkuil wrote: post #10
38 wrote: post #9

Earlier quoted context omitted.

> protobuf such an awful format, I wish people would stop using it

Why? The format is not that bad. The binding/libraries OTOH are often awful and they often require unnecessary full-message deserialization

Protobuf is a tag-length-value (TLV) encoding. It's bad. TLV is the thing that everyone loves to hate about ASN.1's DER.

Re: BuffDB is a Rust library to simplify multi-plexing on edge devices

#13

Post by cryptonector »

38 wrote: post #11
ithkuil wrote: post #10

Earlier quoted context omitted.

Why? The format is not that bad. The binding/libraries OTOH are often awful and they often require unnecessary full-message deserialization

not self describing. if it was just the field names I could deal with that, but even the values are ambiguous, since the same type is used for Bytes and embedded Messages. the worst part is the wire type integer has two unused values, so they easily could have added a wire type for embedded messages

Self-describing is point-less for serializations. There is a great deal of history here. ASN.1 has self-describing encoding rules such as BER/DER/CER, XER (XML), JER (JSON), and GSER (never mind), and it has non-self-describing serializations like PER (packed encoring rules) and OER (octet encoding rules). XML and JSON are self-describing, naturally. FastInfoSet is a PER-based fast encoding for XML, because it turns out that XML is slow (imagine that). XDR is a non-self-describing serialization format that resembles OER but with 4-octet alignment. Flat buffers is essentially an OER-ish encoding for the same IDL as protobufs, and is much better than protobufs.

It would be nice if the next serialization format either is truly original or just solves problems that somehow none of the many existing schemes do.

How many serialization formats are there? See: https://en.wikipedia.org/wiki/Comparison_of_data-serializati... (which is NOT a complete list).

Re: BuffDB is a Rust library to simplify multi-plexing on edge devices

#14

Post by jakjak123 »

38 wrote: post #9
jakjak123 wrote: post #5

Is it a protobuf based database store, or is it a database that uses grpc as its connection communication? Could be a bit clearer from the frontpage

> protobuf such an awful format, I wish people would stop using it

I think it has many good parts. There are waaaaay worse formats I have worked with over the years. Like COM, java RMI, many variants of SOAP, handcrafted json in countless variants…

Re: BuffDB is a Rust library to simplify multi-plexing on edge devices

#15

Post by 38 »

cryptonector wrote: post #13
38 wrote: post #11

Earlier quoted context omitted.

not self describing. if it was just the field names I could deal with that, but even the values are ambiguous, since the same type is used for Bytes and embedded Messages. the worst part is the wire type integer has two unused values, so they easily could have added a wire type for embedded messages

Self-describing is point-less for serializations. There is a great deal of history here. ASN.1 has self-describing encoding rules such as BER/DER/CER, XER (XML), JER (JSON), and GSER (never mind), and it has non-self-describing serializations like PER (packed encoring rules) and OER (octet encoding rules). XML and JSON are self-describing, naturally. FastInfoSet is a PER-based fast encoding for XML, because it turns…

> Self-describing is point-less for serializations

you couldn't be more wrong. what happens when you lose the schema, or never had access to it in the first place? think from the point of view of reverse engineering

Re: BuffDB is a Rust library to simplify multi-plexing on edge devices

#16

Post by cryptonector »

38 wrote: post #15
cryptonector wrote: post #13

Earlier quoted context omitted.

Self-describing is point-less for serializations. There is a great deal of history here. ASN.1 has self-describing encoding rules such as BER/DER/CER, XER (XML), JER (JSON), and GSER (never mind), and it has non-self-describing serializations like PER (packed encoring rules) and OER (octet encoding rules). XML and JSON are self-describing, naturally. FastInfoSet is a PER-based fast encoding for XML, because it turns…

> Self-describing is point-less for serializations you couldn't be more wrong. what happens when you lose the schema, or never had access to it in the first place? think from the point of view of reverse engineering

Why would you lose it? NFS never lost its XDR schema, for example. Do you have any examples where the schema got lost?

Re: BuffDB is a Rust library to simplify multi-plexing on edge devices

#17

Post by eddd-ddde »

38 wrote: post #15
cryptonector wrote: post #13

Earlier quoted context omitted.

Self-describing is point-less for serializations. There is a great deal of history here. ASN.1 has self-describing encoding rules such as BER/DER/CER, XER (XML), JER (JSON), and GSER (never mind), and it has non-self-describing serializations like PER (packed encoring rules) and OER (octet encoding rules). XML and JSON are self-describing, naturally. FastInfoSet is a PER-based fast encoding for XML, because it turns…

> Self-describing is point-less for serializations you couldn't be more wrong. what happens when you lose the schema, or never had access to it in the first place? think from the point of view of reverse engineering

I imagine when someone is choosing a serialization solution they probably don't really care about people trying to reverse engineer it... And if they did, they would just make schemas available instead.

And if you lose your own schema, then you probably have more serious underlying problems.

Re: BuffDB is a Rust library to simplify multi-plexing on edge devices

#18

Post by jamil7 »

38 wrote: post #9
jakjak123 wrote: post #5

Is it a protobuf based database store, or is it a database that uses grpc as its connection communication? Could be a bit clearer from the frontpage

> protobuf such an awful format, I wish people would stop using it

What's an alternative you would recommend?

Re: BuffDB is a Rust library to simplify multi-plexing on edge devices

#20

Post by ithkuil »

cryptonector wrote: post #12
ithkuil wrote: post #10

Earlier quoted context omitted.

Why? The format is not that bad. The binding/libraries OTOH are often awful and they often require unnecessary full-message deserialization

Protobuf is a tag-length-value (TLV) encoding. It's bad. TLV is the thing that everyone loves to hate about ASN.1's DER.

> It's bad

I'm not sure that's a helpful way to look at things.

There are tradeoffs. Can you elaborate more about what aspects of a TLV encoding you find problematic? Is it decoding speed? The need to copy the encoded value into a native value in order to make use of it? Something else?

Post reply on HN