Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FlexBuffers

google.github.io

Post reply on HN
Print view View original

131–140 of 166 posts

Re: FlexBuffers

#131

Post by awild »

secondcoming wrote: post #126

I'm looking to replace our current Boost.Serialization code with something else. I have the following requirements: - serialisation cannot be intrusive (i.e. I don't want a class definition to be generated from a schema file) - zero copy - some sort of versioning so any accidental message version mismatch between sender and receiver doesn't cause a crash / undefined behviour. - many language support Protobufs are int…

My predecessor has employed smile in all our m2m communication. As far as I have understood it, it aims to be a binary representation of json. I cannot speak for its speed but I trust him to a certain extent.

Re: FlexBuffers

#132

Post by kentonv »

jeffbee wrote: post #72
kentonv wrote: post #15

Earlier quoted context omitted.

This is incorrect -- it is not possible to implement Protobuf in a way that achieves the notion of "zero-copy" that Cap'n Proto and FlatBuffers achieve. This probably comes down to a disagreement on what "zero-copy" means. Some people use the term "zero-copy" to mean only that when the message contains a string or byte array, the parsed representation of those specific fields will point back into the original message…

Nothing of your comment was false but there's no intrinsic value to your stronger definition of zero-copy. A direct mapping to memory is not always optimal for performance. Indeed, there are high-performance computation packages that compress data structures in L1-cached-sized blocks, to save main memory bandwidth. So, you've used the word "achieve" to decorate an outcome that might not be optimal. By the way I worke…

> no intrinsic value to your stronger definition of zero-copy.

Whoa, that's a very strong statement. But then the rest of the paragraph gets a lot weaker.

> there are high-performance computation packages that compress data structures in L1-cached-sized blocks

This seems like a non sequitur. Of course hand-tuned data structures can achieve higher performance than any serialization framework, but what does that have to do with zero-copy vs. protobuf? Are you suggesting that protobuf encoding would be a good choice for these people?

> So, you've used the word "achieve" to decorate an outcome that might not be optimal.

I'm not sure why "achieve" would imply "optimal". Of course whether this is an advantage depends on the use case.

There are many cases where zero-copy doesn't provide any real advantages. If you're just sending messages over a standard network socket, then yeah, zero-copy probably isn't going to make things faster. There are already several copies inherent in network communication.

But if you have a huge protobuf file on disk and you want to read one field in the middle of it, that's just not something you can do in any sort of efficient way. With zero-copy, you can do this trivially with mmap().

Or if you're communicating over shared memory between processes on the same machine, then the entire serialize/parse round trip required with protobuf is 100% waste. Zero-copy would let you build and consume the structure from the same memory pages.

These seem "intrinsically valuable"?

> we could never get flatbuffers to go any faster.

What use case were you testing? Did you test any zero-copy serializations other than flatbuffers?

I've heard from lots of people that say Cap'n Proto beat Protobuf in their tests... but it definitely depends on the use case.

Re: FlexBuffers

#133

Post by kentonv »

jeffbee wrote: post #123
cmrdporcupine wrote: post #117

Earlier quoted context omitted.

Not-Faster on what platform? In Borg, in Google3 code, deployed on a fast machine with a nice fast wide memory bus and a large cache? What about in embedded code, or in a game? A place where memory bandwidth is scarce, or where we're trying desperately to reduce the number of syscalls and jumps back and forth between kernel and user space? Having the entire payload memory mapped, and copies avoided, makes an absolute…

> Not-Faster on what platform? In Borg, in Google3 code, deployed on a fast machine with a nice fast wide memory bus and a large cache? Weird rebuttal, considering that my argument is that the copying nature of protobuf can save memory bandwidth.

> my argument is that the copying nature of protobuf can save memory bandwidth.

Huh? An extra pass over the data to parse it obviously uses more memory bandwidth.

I might understand your argument if parsing converted the data into a memory-bandwidth-optimized format, but the protobuf parsed form is certainly not that, unless things have changed very drastically since I worked on it.

Re: FlexBuffers

#134

Post by jeffbee »

kentonv wrote: post #132
jeffbee wrote: post #72

Earlier quoted context omitted.

Nothing of your comment was false but there's no intrinsic value to your stronger definition of zero-copy. A direct mapping to memory is not always optimal for performance. Indeed, there are high-performance computation packages that compress data structures in L1-cached-sized blocks, to save main memory bandwidth. So, you've used the word "achieve" to decorate an outcome that might not be optimal. By the way I worke…

> no intrinsic value to your stronger definition of zero-copy. Whoa, that's a very strong statement. But then the rest of the paragraph gets a lot weaker. > there are high-performance computation packages that compress data structures in L1-cached-sized blocks This seems like a non sequitur. Of course hand-tuned data structures can achieve higher performance than any serialization framework, but what does that have t…

> Or if you're communicating over shared memory between processes on the same machine, then the entire serialize/parse round trip required with protobuf is 100% waste.

This is the part I don’t agree with. What I’m saying there is value in using encoded structures not only between servers and not only over shared memory but even within a single process. Yes, you discard the ability to just jump to any random field, but that is not always important. Often it can be better to spend some compute cycles and L1 accesses to save main memory accesses. If you are having to make full access to some kind of data anyway, then packing it makes a ton of sense. Consider any kind of delta-encoded column of values ... you can’t seek within it, but if the deltas are smaller than the absolutes, this can save massive amounts of main memory bandwidth. This is why I argue that representing something as a C struct in main memory is not obviously advantageous, outside some given workloads.

As for flatbuf at google I’m sure you’re aware that the only way to get the kind of mindshare you’d need to ship it would be to make websearch measurably faster.

Re: FlexBuffers

#135

Post by kentonv »

anonymoushn wrote: post #114
kentonv wrote: post #57

Earlier quoted context omitted.

Heh, well, you could get it for free in Cap'n Proto if Cap'n Proto allowed pointer aliasing. It doesn't, though, because if it did, then messages would not be trees, they'd be graphs, which ruins a lot of stuff. For example, a very common thing to do with a message is copy one branch of the tree into a different message. Deep-copying a branch of a tree is easy. Deep-copying a branch of a graph, though -- what does th…

Copying a branch of a DAG has about the same meaning as copying a branch of a tree, right?

For a DAG, maybe, but it requires a lot more bookkeeping. Now you have to remember all the pointers you've seen before in order to detect dupes. To do that you probably need a hash map and some dynamic memory allocation, ugh.

And what happens if you copy two different branches of one message into another, and they happen to share some children? Do you have to keep your seen-pointer map around across multiple copies?

For a cyclic graph, things get more confusing. Copying one branch of a fully-connected cyclic graph always means copying the entire graph. Apps can easily get into trouble here. Imagine an app that implements its own tree structure where nodes have "parent" pointers. If they try to copy one branch into another message, they accidentally copy the entire tree (via the parent pointers) and might not even realize it.

The one way that I think pointer aliasing could be made to work is if pointers that are allowed to alias are specially-marked, and are non-owning. So each object still has exactly one parent object, but might have some other pointers pointing to it from elsewhere. A copy would not recurse into these pointers; it would only update them if the target object happened to be part of the copy, otherwise they would have to become null.

But I haven't yet had any reason to try implementing this approach. And apps can get by reasonably well without it, by using integer indexes into a table.

Re: FlexBuffers

#136

Post by convolvatron »

uluyol wrote: post #3

So here's how I think this fits into all the other different types of data serialization: Schema-ful, copying: Protobuf, Thrift, plenty more Schema-ful, zero-copy: Cap'n'proto, Flatbuffers Schema-less, copying: Json (binary and other variants included), XML Schema-less, zero-copy: Flexbuffers (Any others? This seems new to me)

but i really do wonder why this is such a matter of debate. if performance and specific semantics are an issue...just use the standard tricks and write bytes into a buffer and push it onto the wire.

if performance isn't an issue, then just use any of these tools. unless the tooling cost and representational issues make it easier to to just use bytes.

all for abstractions..but people seem to to be blind to the idea that there's a perfectly good one a short step down from capn proto

Re: FlexBuffers

#137

Post by fluffything »

jeffbee wrote: post #134
kentonv wrote: post #132

Earlier quoted context omitted.

> no intrinsic value to your stronger definition of zero-copy. Whoa, that's a very strong statement. But then the rest of the paragraph gets a lot weaker. > there are high-performance computation packages that compress data structures in L1-cached-sized blocks This seems like a non sequitur. Of course hand-tuned data structures can achieve higher performance than any serialization framework, but what does that have t…

> Or if you're communicating over shared memory between processes on the same machine, then the entire serialize/parse round trip required with protobuf is 100% waste. This is the part I don’t agree with. What I’m saying there is value in using encoded structures not only between servers and not only over shared memory but even within a single process. Yes, you discard the ability to just jump to any random field, bu…

> Yes, you discard the ability to just jump to any random field, but that is not always important.

I don't think this is the criticism being raised.

The main criticism being raised against non-zero-copy serialization is that this often requires maintaining different memory representations for the same value - the copies are the consequence of transforming from one representation to another one.

Re: FlexBuffers

#138

Post by jeffbee »

fluffything wrote: post #137
jeffbee wrote: post #134

Earlier quoted context omitted.

> Or if you're communicating over shared memory between processes on the same machine, then the entire serialize/parse round trip required with protobuf is 100% waste. This is the part I don’t agree with. What I’m saying there is value in using encoded structures not only between servers and not only over shared memory but even within a single process. Yes, you discard the ability to just jump to any random field, bu…

> Yes, you discard the ability to just jump to any random field, but that is not always important. I don't think this is the criticism being raised. The main criticism being raised against non-zero-copy serialization is that this often requires maintaining different memory representations for the same value - the copies are the consequence of transforming from one representation to another one.

We do that all the time in high-performance computing. You keep a packed representation in memory and unpack it in small pieces to operate on it. Sparse matrices, compressed columns, etc. This is not evil, it’s an adaptation to the way the machine works. Saying that Kenton’s definition of zero-copy is unconditionally better is an aesthetic argument and I don’t buy it.

Re: FlexBuffers

#139

Post by jeffbee »

kentonv wrote: post #133
jeffbee wrote: post #123

Earlier quoted context omitted.

> Not-Faster on what platform? In Borg, in Google3 code, deployed on a fast machine with a nice fast wide memory bus and a large cache? Weird rebuttal, considering that my argument is that the copying nature of protobuf can save memory bandwidth.

> my argument is that the copying nature of protobuf can save memory bandwidth. Huh? An extra pass over the data to parse it obviously uses more memory bandwidth. I might understand your argument if parsing converted the data into a memory-bandwidth-optimized format, but the protobuf parsed form is certainly not that, unless things have changed very drastically since I worked on it.

Right but the parsed form is exactly what I meant when I said it’s an aspect of the implementation. The generated C++ code you get from Google’s protoc is a sparse thing, to be sure. But that is an artifact of the implementation. You can do anything you want with a protobuf and there are numerous independent implementations in the wild, in many languages.

Re: FlexBuffers

#140

Post by fluffything »

jeffbee wrote: post #138
fluffything wrote: post #137

Earlier quoted context omitted.

> Yes, you discard the ability to just jump to any random field, but that is not always important. I don't think this is the criticism being raised. The main criticism being raised against non-zero-copy serialization is that this often requires maintaining different memory representations for the same value - the copies are the consequence of transforming from one representation to another one.

We do that all the time in high-performance computing. You keep a packed representation in memory and unpack it in small pieces to operate on it. Sparse matrices, compressed columns, etc. This is not evil, it’s an adaptation to the way the machine works. Saying that Kenton’s definition of zero-copy is unconditionally better is an aesthetic argument and I don’t buy it.

You are still not getting it.

They are not talking about "unpacking on the fly for processing", but rather about "unpacking on memory to be able to call an opaque API outside your control that expects the unpacked representation". That requires copying in-memory to interface with that API.

Your approach only works if you are willing to "re-implement the world" to interface with whatever packed format suits your application.

With zero-copy serialization you don't have to do that.

Post reply on HN