FlexBuffers

uluyol wrote: post #3

So here's how I think this fits into all the other different types of data serialization: Schema-ful, copying: Protobuf, Thrift, plenty more Schema-ful, zero-copy: Cap'n'proto, Flatbuffers Schema-less, copying: Json (binary and other variants included), XML Schema-less, zero-copy: Flexbuffers (Any others? This seems new to me)

Sounds like some implementations of MessagePack may be in the same category.

Re: FlexBuffers

#6

Post by cortesoft » Sun, Jun 21, 2020, 1:20 AM UTC

uluyol wrote: post #3

So here's how I think this fits into all the other different types of data serialization: Schema-ful, copying: Protobuf, Thrift, plenty more Schema-ful, zero-copy: Cap'n'proto, Flatbuffers Schema-less, copying: Json (binary and other variants included), XML Schema-less, zero-copy: Flexbuffers (Any others? This seems new to me)

Aren't the newer versions of protobuf zero-copy?

Re: FlexBuffers

#7

Post by yablak » Sun, Jun 21, 2020, 1:22 AM UTC

cortesoft wrote: post #6

uluyol wrote: post #3

So here's how I think this fits into all the other different types of data serialization: Schema-ful, copying: Protobuf, Thrift, plenty more Schema-ful, zero-copy: Cap'n'proto, Flatbuffers Schema-less, copying: Json (binary and other variants included), XML Schema-less, zero-copy: Flexbuffers (Any others? This seems new to me)

Aren't the newer versions of protobuf zero-copy?

Now that absl::Cord has been opensourced, I'd expect a release of protobuf that takes advantage of it. Up to you to use the API though.

Re: FlexBuffers

#8

Post by mmastrac » Sun, Jun 21, 2020, 1:26 AM UTC

ccktlmazeltov wrote: post #4

there's a lot of C++ around this FlexBuffer thing so I'm not sure how relevant it is outside of C++. Any idea?

We use FlatBuffers (the parent project) in our code @ FS... that means it has to work in C++, Rust, Java, and TypeScript. It's mostly the schema processing that requires a C++ compiler to make it work.

It's a pretty stellar cross-platform serialization toolkit and the zero-copy support for reading is no joke.

Re: FlexBuffers

#9

Post by jchook » Sun, Jun 21, 2020, 1:30 AM UTC

ccktlmazeltov wrote: post #4

there's a lot of C++ around this FlexBuffer thing so I'm not sure how relevant it is outside of C++. Any idea?

AFAIK some languages can use C++ fairly easily, including D and Rust

Re: FlexBuffers

#10

Post by ChristianBundy » Sun, Jun 21, 2020, 1:37 AM UTC

uluyol wrote: post #3

So here's how I think this fits into all the other different types of data serialization: Schema-ful, copying: Protobuf, Thrift, plenty more Schema-ful, zero-copy: Cap'n'proto, Flatbuffers Schema-less, copying: Json (binary and other variants included), XML Schema-less, zero-copy: Flexbuffers (Any others? This seems new to me)

> Any others?

Would BIPF fall into this category? It's a serialization format for JSON: https://github.com/dominictarr/bipf