Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RPC Olympics – The Search for the Perfect RPC Protocol

perimeterx.com

Post reply on HN
Print view View original

31–40 of 71 posts

Re: RPC Olympics – The Search for the Perfect RPC Protocol

#31

Post by hhas01 »

treve wrote: post #23
alfalfasprout wrote: post #14

Having just looked into this recently, I can also safely say that moving to REST HTTP/2 on its own already provides a significant speed benefit. Closing the gap to GRPC (or even substantially beating it) is possible by switching to a fast serialization format (eg; flatbuffers, msgpack, capnproto). While GRPC has its place it also comes with headaches like pretty lousy generated interfaces, horrible debuggability, and…

It's kinda weird to compare REST to RPC protocols though. It's a completely different paradigm. Maybe you're just building a HTTP API without the REST part? (JSON-RPC is a thing)

Indeed; true REST interactions tend to be quite coarse-grained; essentially requests to update a remote state machine to match the described state. Most of what gets called “REST” isn’t; it’s just ad-hoc RPC sent over HTTP with arguments encoded as XML/JSON, which is probably what parent really means. Take casual claims of RESTfulness with the requisite bucket of salt. (If the term “REST API” is used, you can toss the bucket entirely as the very phrase is itself an oxymoron.)

Re: RPC Olympics – The Search for the Perfect RPC Protocol

#32

Post by hhas01 »

pkage wrote: post #2

Strong recommendation for ZeroMQ as well. It's a thin layer on TCP, is very fast, offers bindings in many languages, and is fairly flexible as well. https://zeromq.org/

Surprised article didn’t include this in its comparison. Seems an obvious contender.

Re: RPC Olympics – The Search for the Perfect RPC Protocol

#33

Post by a_t48 »

silasb wrote: post #19

I recently spent some time between raw UDP and GRPC and learned a ton. This was for a hardware project so it's different from what web devs typically work on. The most important thing that I found out while doing my research is it's not the fastest bytes that win. Well, that does matter, but it's not that important. It's reducing performance variance [1]. While my project wasn't optimizing for speed, it was optimizin…

At my last job, I had a need for high bandwidth low latency messaging over UDP - we started out with raw C structs over a home built reliable UDP library. We ended up laying protobuf on top a year after because it was honestly a headache not having a serialization library as message types got more complex. It didn't end up causing enough slowdown to impact the run speed of the application (all serialization was done on the app thread rather than the very performance sensitive network thread) as we had other bottlenecks, and the developer workflow (and additional message validation) made it worth it. SBE looks neat, though, I didn't know about that one.

Re: RPC Olympics – The Search for the Perfect RPC Protocol

#34

Post by e12e »

treve wrote: post #23
alfalfasprout wrote: post #14

Having just looked into this recently, I can also safely say that moving to REST HTTP/2 on its own already provides a significant speed benefit. Closing the gap to GRPC (or even substantially beating it) is possible by switching to a fast serialization format (eg; flatbuffers, msgpack, capnproto). While GRPC has its place it also comes with headaches like pretty lousy generated interfaces, horrible debuggability, and…

It's kinda weird to compare REST to RPC protocols though. It's a completely different paradigm. Maybe you're just building a HTTP API without the REST part? (JSON-RPC is a thing)

Actually traditional RPC is one of the paradigms considered in Fielding's thesis coining the term. I don't know why people seem to insist there are no other models than REST if you happen to have browser on one end. Sending Javascript down the asynchronous pipe designed for XML was at the very least a step back away from REST toward something more traditional (moving code, not data).

Section 3.5 "Moving code styles" (and 3.4 for RPC):

https://www.ics.uci.edu/~fielding/pubs/dissertation/net_arch...

Re: RPC Olympics – The Search for the Perfect RPC Protocol

#36

Post by philsnow »

ngneer wrote: post #21
JoshTriplett wrote: post #18

Earlier quoted context omitted.

Be careful using the pickle format; it's not considered safe to unpickle untrusted data. https://docs.python.org/3/library/pickle.html

You beat me to it. Sounds like you would be opening yourself up to a variant of CSRF. One user could upload untrusted data that would be fed into an unsuspecting user. You should never feed or consume an untrusted pickle.

Pickles can also be time bombs, especially around python upgrades. Sometimes (ok, rarely) the serialization / deserialization of some types changes between versions of python.

Another issue is painting yourself into a corner: when you use pickles, you make it harder to either switch away from python in the future or consume the same serialized objects from any non-python (micro)service. This can delay or prevent transitions away from python that would otherwise make sense.

Re: RPC Olympics – The Search for the Perfect RPC Protocol

#37

Post by thedance »

thedance wrote: post #28
parhamn wrote: post #15

Cool! I don't know if a comparison of these really makes sense. You'd be conflating transport, serialization, and stream format (unary/stream). For fullstack dev, I've been immensely happy with grpc/protobuf over http because of the type safety I get communicating between Golang and Typescript. This eliminates a whole class of bugs but is only a serialization benefit. Generally: use the thing with the best tooling!

There is no type safety in grpc or any other protobuf RPC scheme, full stop. The recipient of a message makes an assumption about the meaning of the message and decodes it accordingly. Any encoded protobuf might successfully decode as any other protobuf. To repeat, grpc has no "type safety" whatsoever.

Quick reminder that rebuttals are more valuable than downvotes.

Re: RPC Olympics – The Search for the Perfect RPC Protocol

#38

Post by H8crilA »

Apples to oranges.

TCP has no concept of a message, you need to build it on top of TCP.

Which HTTP does, but it doesn't have a standard concept of a message format, you need to build it top of HTTP.

Which gRPC does, but it closes the connection after a successful exchange.

Which is avoided by streaming gRPC - makes sense if you know you'll be talking more over this channel.

None of those protocols are really comparable, and most of the differences boil down to the serialisation protocol (binary/proto or textual, like JSON).

Re: RPC Olympics – The Search for the Perfect RPC Protocol

#39

Post by tweenagedream »

H8crilA wrote: post #38

Apples to oranges. TCP has no concept of a message, you need to build it on top of TCP. Which HTTP does, but it doesn't have a standard concept of a message format, you need to build it top of HTTP. Which gRPC does, but it closes the connection after a successful exchange. Which is avoided by streaming gRPC - makes sense if you know you'll be talking more over this channel. None of those protocols are really comparab…

Usually grpc connections are left open after a message is sent and received. And since it's based on h2, multiple message streams can be multiplexed over a single connection. That is part of the benefit of grpc, establish a long lived connection and send your messages as needed. Similar to using keep-alive with http.

Re: RPC Olympics – The Search for the Perfect RPC Protocol

#40

Post by nerdponx »

nurettin wrote: post #8
pkage wrote: post #2

Strong recommendation for ZeroMQ as well. It's a thin layer on TCP, is very fast, offers bindings in many languages, and is fairly flexible as well. https://zeromq.org/

especially when you use the router pattern on server side in order to have multiple sessions on the same socket.

Got any references on the router pattern? I searched but came up with a lot of IT related articles.
Post reply on HN