101–110 of 135 posts
Performance of Protobuf is a joke. Why not use a zero copy format so that serialization is free ? For example, my format Lite³ which outperforms Google Flatbuffers by 242x: https://github.com/fastserial/lite3
Earlier quoted context omitted.
Besides the other comments already here about code gen & contracts, a bigger one for me to step away from json/xml is binary serialization. It sounds weird, and its totally dependent on your use case, but binary serialization can make a giant difference. For me, I work with 3D data which is primarily (but not only) tightly packed arrays of floats & ints. I have a bunch of options available: 1. JSON/XML, readable, eas…
This use case totally makes sense of course. I’m thinking about why people use Protobuf for their string, uuid and int powered CRUD app.
FlatBuffers are already faster than that. But that's not why we choose Protobuf. It's because a megacorp maintains it.
You're saying we choose Protobufs [1] because Google maintains it but not FlatBuffers [2]? [1] - https://github.com/protocolbuffers/protobuf : Google's data interchange format [2] - https://github.com/google/flatbuffers : Also maintained by Google
It just means a person working at Google used that avenue to open source them.
Google offers a legal few avenues to allow you to open source your stuff while working there but one of the easiest it just to assign copyright to Google and shove it under their GitHub.
It just means a Googler published it, not that Google itself is maintaining it.
I don't know what the status of flatbuffers is specifically, but I can say I never encountered it in use in the 10 years I worked there. (I use it a lot now on my own things post-Google)
Don't read clickbaity headlines and scan hacker news five times faster.
Earlier quoted context omitted.
Given they heavily used LLMs for this optimization, makes you wonder why they didn’t use them to just port the C library to rust entirely. I think the volume of library ports to more languages/the most performant languages is going to explode, especially given it’s a relatively deterministic effort so long as you have good tests and api contracts, etc
The underlying C library interacts directly with the postgres query parser (therefore, Postgres source). So unless you rewrite postgres in Rust, you wouldn't be able to do that.
I agree that LLMs will make clients/interfaces in every language combination much more common, but I wonder the impact it’ll have on these big software projects if more people stop learning C.
What I find particularly ironic is that the title make it feel like Rust gives a 5x performance improvement when it actually slows thing down. The problem they have software written in Rust, and they need to use the libpg_query library, that is written in C. Because they can't use the C library directly, they had to use a Rust-to-C binding library, that uses Protobuf for portability reasons. Problem is that it is slo…
> What I find particularly ironic is that the title make it feel like Rust gives a 5x performance improvement when it actually slows thing down. Rust didn't slow them down. The inefficient design of the external library did. Calling into C libraries from Rust is extremely easy. It takes some work to create a safer wrapper around C libraries, but it's been done for many popular libraries. This is the first and only ti…
Calling the C function is not the problem here. It is dealing with the big data structure this function returns in a Rust-friendly manner.
This is something Protobuf does very well, at the cost of performance.
I find the title a bit misleading. I think it should be titled It’s Faster to Copy Memory Directly than Send a Protobuf. Which then seems rather obvious that removing a serialization and deserialization step reduces runtime.
TIL serializing a protobuf is only 5 times slower than copying memory, which is way faster than I thought it’d be. Impressive given all the other nice things protobuf offers to development teams.