31–40 of 135 posts
"5 times faster" reminds me of Cap'n Proto's claim: in benchmarks, Cap’n Proto is INFINITY TIMES faster than Protocol Buffers: https://capnproto.org/
I mean, cap'n'proto is written by the same person who created protobuf, so they are legit (and that somewhat jokish claim is simply that it requires no parsing).
Notably, Protobuf 2, a rewrite of Protobuf 1. Protobuf 1 was created by Sanjay Ghemawat, I believe.
Just for fun, how often do regular-sized companies that deal in regular-sized traffic need Protobuf to accomplish their goals in the first place, compared to JSON or even XML with basic string marshalling?
Having a way to describe your whole API and generate bindings is a godsend. Yes, it can be done with JSON and OpenApi, yet it’s not mandatory.
I wrote assembly, memory mapping oriented protobuf software... in assembly, then what? I am allowed to say I am going 1000 times faster than rust now???
Earlier quoted context omitted.
> I vaguely recall that there's a Rust macro to automatically convert recursive functions to iterative. Isn't that just TCO or similar? Usually a part of the compiler/core of the language itself, AFAIK.
I haven't been following become/TCO in Rust - but what I've usually seen is TCO getting flipped off because it interferes with backtraces and debugging. So I think there's value in providing it as an explicit opt-in; that way when you're reading the code, you know to account for it when you're looking at backtraces. Additionally, if you're relying on TCO it might be a major bug if the compiler isn't able to apply it…
Suppose I have a recursive function f(n: u8) where f(0) is 0 and otherwise f(n) is n * bar(n) + f(n-1)
I might well write that with a local temporary to calculate bar(n) and then we do the sum, but this would inhibit TCO because that temporary should exist after we did the recursive calculation, even though it doesn't matter in practice.
A compiler could try to cleverly figure out whether it matters and destroy that local temporary earlier then apply TCO, but now your TCO is fragile because a seemingly minor code change might fool that "clever" logic, by ensuring it isn't correct to make this change and breaking your optimisation.
The `become` keyword is a claim by the programmer that we can drop all these locals and do TCO. So because the programmer claimed this should work they're giving the compiler permission to attempt the early drop and if it doesn't work and can't be TCO then complain that the program is wrong.
tldr: they replaced using protobuf as the type system across language boundaries for FFI with true FFI
I loved, every clickbait title should come with a tldr just like this one.
I find the title a bit misleading. I think it should be titled It’s Faster to Copy Memory Directly than Send a Protobuf. Which then seems rather obvious that removing a serialization and deserialization step reduces runtime.
Just doing memcpy or mmap would be even faster. But the same Rust advocates bragging about Rust speed frown upon such unsecure practices in C/C++.
Just for fun, how often do regular-sized companies that deal in regular-sized traffic need Protobuf to accomplish their goals in the first place, compared to JSON or even XML with basic string marshalling?
It sounds weird, and its totally dependent on your use case, but binary serialization can make a giant difference.
For me, I work with 3D data which is primarily (but not only) tightly packed arrays of floats & ints. I have a bunch of options available:
1. JSON/XML, readable, easy to work with, relatively bulky (but not as bad as people think if you compress) but no random access, and slow floating point parsing, great extensibility.
2. JSON/XML + base64, OK to work with, quite bulky, no random access, faster parsing, but no structure, extensible.
3. Manual binary serialization: hard to work with, OK size (esp compressed), random access if you put in the effort, optimal parsing, not extensible unless you put in a lot of effort.
4. Flatbuffers/protobuf/capn-proto/etc: easy to work with, great size (esp compressed), random access, close-to-optimal parsing, extensible.
Basically if you care about performance, you would really like to just have control of the binary layout of your data, but you generally don't want to design extensibility and random access yourself, so you end up sacrificing explicit layout (and so some performance) by choosing a convenient lib.
We are a very regularly sized company, but our 3D data spans hundreds of terabytes.
(also, no, there is no general purpose 3D format available to do this work, gltf and friends are great but have a small range of usecases)