121–130 of 135 posts
Earlier quoted context omitted.
"never" huh?
Pretty much. The tide of rust evangelism has been turned in favor of complainers for a while now. Nothing compared to JS and React hate, but still.
Earlier quoted context omitted.
Google loves to reinvent shit because they didn't understand it. And to get promo. In this case, ASN.1. And protobufs are so inefficient that they drive up latency and datacenter costs, so they were a step backwards. Good job, Sanjay.
Really dismissive and ignorant take from a bystander. Back it up with your delivery that does better instead of shouting with a pitchfork for no reason.
Earlier quoted context omitted.
> I mean, cap'n'proto is written by the same person who created protobuf Notably, Protobuf 2, a rewrite of Protobuf 1. Protobuf 1 was created by Sanjay Ghemawat, I believe.
Google loves to reinvent shit because they didn't understand it. And to get promo. In this case, ASN.1. And protobufs are so inefficient that they drive up latency and datacenter costs, so they were a step backwards. Good job, Sanjay.
Earlier quoted context omitted.
Really dismissive and ignorant take from a bystander. Back it up with your delivery that does better instead of shouting with a pitchfork for no reason.
This bystander has been using protobufs for more than ten years. I'm not sure what I need to deliver since ASN.1, Cap'n Proto and Flatbuffers are all more efficient and exist already. ASN.1 was on the scene in 1984 and was already more efficient than protobufs.
Can someone explain how protobuf ended up in the middle here? I'm just totally confused; the C ABI exists in almost every language, why did they need protobuf here?
I had experience with writing safe bindings to structures created in C library, and it is a real pain. You spend a lot of times reverse engineering C code to get an idea of the intent of those who had wrote the code. You need to know which pointers can address the same memory. You need to know which pointers can be NULL or just plain invalid. You need to know which pointers you get from C code or pass to it along with ownership, and which are just borrowed. It maybe (and often is) unclear from the documentation, so you are going to read a lot of C code, trying to guess what the authors were thinking when writing it. Generating hypotheses about the library behavior (like 'library never does THIS with the pointer') and trying to prove them by finding all the code dealing with the pointer.
It can be easy in easy situations, or it can be really tricky and time consuming. So it can make sense to just insert serialization/deserialization to avoid dealing with C code.
Earlier quoted context omitted.
>> But had they written their software in C, they wouldn't have needed to do any conversion at all. It means they could have titled the article "How we lowered the performance penalty of using Rust". That's not really fair. The library was doing serialization/deserialization which was poor design choice from a performance perspective. They just made a more sane API that doesn't do all that extra work. It might best b…
> BTW what makes you think writing their end in C would yield even higher performance? C is not inherently faster, you are right about that. But what I understand is that the library they use works with data structures that are designed to be used in a C-like language, and are presumably full of raw pointers. These are not ideal for working in Rust, instead, presumably, they wrote their own data model in Rust fashion…
One would think. But since caches have grown so large, and memory speed and latency haven't scaled with compute, so long as the conversion fits in the cache and is operating on data already in the cache from previous operations, which admittedly takes some care, there's often an embarrassing amount of compute sitting idle waiting for the next response from memory. So if your workload is memory or disk or network bound, conversions can oftentimes be "free" in terms of wall clock time. At the cost of slightly more wattage burnt by the CPU(s). Much depends on the size and complexity of the data structure.
I find the title a bit misleading. I think it should be titled It’s Faster to Copy Memory Directly than Send a Protobuf. Which then seems rather obvious that removing a serialization and deserialization step reduces runtime.
TIL serializing a protobuf is only 5 times slower than copying memory, which is way faster than I thought it’d be. Impressive given all the other nice things protobuf offers to development teams.
> Protobuf performs up to 6 times faster than JSON. - https://auth0.com/blog/beating-json-performance-with-protobu... (2017)
That's a 30x faster just by switching to a zero-copy data format that's suitable for both in memory use and network. JSON services spend 20-90% of their compute on serde. A zero copy data format would essentially eliminate it.
Earlier quoted context omitted.
This bystander has been using protobufs for more than ten years. I'm not sure what I need to deliver since ASN.1, Cap'n Proto and Flatbuffers are all more efficient and exist already. ASN.1 was on the scene in 1984 and was already more efficient than protobufs.
Protobuf has far better ergonomics than ASN.1. ASN.1 is an overcomplicated design-by-committee mess. Backwards compatibility in particular is much harder.
Earlier quoted context omitted.
But something more modern that doesn’t have the encoding/decoding penalty of Protobuf would be better (eg cap’n’proto but there’s a bunch now in this space).
Not that you are wrong, but in the real world this is not significant for most uses. If it is significant you are doing too much IPC. Or maybe using protobuf where you should be making a direct function call. Fix the architecture either way. (similar to how I can make bubble sort faster with careful machine code optimization, but it is hard to make modern tim sort slower in the real world no matter how bad the implem…
Examples that can noticeably slow things down even for “normal” web apps: map types and deeply nested messages.
Earlier quoted context omitted.
Protobuf has far better ergonomics than ASN.1. ASN.1 is an overcomplicated design-by-committee mess. Backwards compatibility in particular is much harder.
I don't doubt your experience, but with X.509 having evolved substantially, and ASN.1 on billions (if not tens of billions) of devices, in practice it seems OK. And it was formally verified early.
By all means, keep using it, but it might be worth figuring out why other people don’t. Hint: it’s not because they’re more stupid than you or are looking to get promoted by big G.
(Personally, I like the ideas and binary encoding behind Capn Proto more than all the alternatives)