Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

gRPC: Internet-scale RPC framework is now 1.0

cloudplatform.googleblog.com

Post reply on HN
Print view View original

101–110 of 111 posts

Re: gRPC: Internet-scale RPC framework is now 1.0

#101

Post by euyyn »

raspasov wrote: post #87
euyyn wrote: post #85

Earlier quoted context omitted.

What is your use case, and what issues did you find?

Incomplete or completely lacking documentation for Objective C and Java, weird bugs, random disconnects. Overall it feels like the Ruby on Rails of networking - an opinionated package/framework that tries to do too much. Also, the whole concept of "as easy as a a local functional call" is a flawed, leaky abstraction.

I'm responsible for the Objective-C part, so please let me know of anything we can improve there. We have a couple of tutorials at http://www.grpc.io/docs/tutorials/ and a quick-start guide at http://www.grpc.io/docs/quickstart/objective-c.html . For bugs and connectivity problems, filing a GitHub issue would be super appreciated.

If you look at the example code, you'll see that RPCs aren't modeled exactly as local function calls. You're right that that wouldn't work very well. The libraries for all or most languages let you make RPCs asynchronously, without blocking the thread. And all of them provide with ways to write and read RPC metadata (headers and trailers).

Re: gRPC: Internet-scale RPC framework is now 1.0

#102

Post by kbaker »

heavenlyhash wrote: post #96
kajecounterhack wrote: post #29

Earlier quoted context omitted.

Performance and versioning are two large benefits. Performance benefit comes from the fact that schema is defined on each side (generally server / server) so you only send the information bytes. With a good RPC system you can also access specific fields of your structure without unpacking (or very fast unpacking, depends on what RPC system you're using). gRPC uses Google's protocol buffers. https://developers.google.…

I'd really love to see something like gRPC implemented over CBOR. CBOR -- Concise Binary Object Notation -- http://cbor.io/ -- is all the performance of a binary protocol, with semantics basically identical to JSON. I appreciate some of the things protobuf does to help you version, but I also do not appreciate the protobuf compiler as a dependency and a hurdle for contributors, or for wire debugging. CBOR has librari…

There are two reasons why I think simpler binary packing libraries like CBOR, MsgPack, or BSON can't really match what Protobuf gives you:

- Assistance with schema evolution and versioning is one of the best parts about using Protobuf in an API. It is really like the best parts of XML and XML Schema (validation, documentation, interoperability) without any of the bloat.

- Working with code generation can be a pain to get working initially, but is very friendly when actually using real objects in code. There is no need to think about any representation on-the-wire... everything 'just works'. There is no need to ensure you don't accidentally serialize fields in the wrong order, or worry about encodings, etc.

Also, there is a binary decoder included with protoc that can print a debug decoding of any protobuf binary message, including integer tags for different fields. Wouldn't you have pretty much the same problems with dissection and debugging on-the-wire in CBOR?

It is really quite pleasant to use Protobuf for an API, I can see why Google is opinionated in including it as the only option with gRPC.

Re: gRPC: Internet-scale RPC framework is now 1.0

#103

Post by areed »

vigilant wrote: post #100

Is gRPC a full fledged server for API calls? e.g.: Will it have things like monitoring (we've handled x calls to this API in the last hour, the average API call took y milliseconds). Clustering? (a client connects to a list of grpc servers, if one server goes down, the client will automatically connect to the next on the list)? And load balancing? If not, are there existing third party tools to implement these, or is…

No, these are separate concerns. I use Kubernetes for the latter two.

Re: gRPC: Internet-scale RPC framework is now 1.0

#104

Post by honkhonkpants »

vigilant wrote: post #100

Is gRPC a full fledged server for API calls? e.g.: Will it have things like monitoring (we've handled x calls to this API in the last hour, the average API call took y milliseconds). Clustering? (a client connects to a list of grpc servers, if one server goes down, the client will automatically connect to the next on the list)? And load balancing? If not, are there existing third party tools to implement these, or is…

As for monitoring, grpc is hooked into census, which provides some rudimentary statistics and is also intended to eventually exfiltrate Dapper tracing out of Google and into the public gRPC user base. It's a bit rudimentary at the moment, but see https://github.com/grpc/grpc/tree/master/src/core/ext/census

Re: gRPC: Internet-scale RPC framework is now 1.0

#105

Post by lyschoening »

Any people using gRPC and Protocol Buffers from Python with good experiences?

The library for Python for one seems very unpythonic to me, starting with all those CamelCase method names. It does not seem possible to use asyncio or any other non-blocking solution on the server.

Finally, gRPC obviously only handles the transport: Are there any other useful related Python packages out there for validation etc.?

Re: gRPC: Internet-scale RPC framework is now 1.0

#106

Post by zellyn »

tokenizerrr wrote: post #9

So how does service discovery work? Where do I read more?

I can speak most directly to Go, where I've been working. You could potentially use one of the maps-to-DNS service discovery systems, although that's limited.

At Square, we created a custom balancer (https://godoc.org/google.golang.org/grpc#Balancer) which not only handles updates from the service discovery system in order to manage the pool of connections, but also handles which connection to use per-call (so we can do targeting of specific capabilities, datacenters, etc.)

Re: gRPC: Internet-scale RPC framework is now 1.0

#107

Post by puzzle »

kajecounterhack wrote: post #29
tptacek wrote: post #25

Something I've wondered for awhile: why would I want to design with gRPC rather than well-defined HTTP/JSON endpoints? Is it just a perf thing?

Performance and versioning are two large benefits. Performance benefit comes from the fact that schema is defined on each side (generally server / server) so you only send the information bytes. With a good RPC system you can also access specific fields of your structure without unpacking (or very fast unpacking, depends on what RPC system you're using). gRPC uses Google's protocol buffers. https://developers.google.…

gRPC can also use FlatBuffers, by the way.

Re: gRPC: Internet-scale RPC framework is now 1.0

#109

Post by heavenlyhash »

kbaker wrote: post #102
heavenlyhash wrote: post #96

Earlier quoted context omitted.

I'd really love to see something like gRPC implemented over CBOR. CBOR -- Concise Binary Object Notation -- http://cbor.io/ -- is all the performance of a binary protocol, with semantics basically identical to JSON. I appreciate some of the things protobuf does to help you version, but I also do not appreciate the protobuf compiler as a dependency and a hurdle for contributors, or for wire debugging. CBOR has librari…

There are two reasons why I think simpler binary packing libraries like CBOR, MsgPack, or BSON can't really match what Protobuf gives you: - Assistance with schema evolution and versioning is one of the best parts about using Protobuf in an API. It is really like the best parts of XML and XML Schema (validation, documentation, interoperability) without any of the bloat. - Working with code generation can be a pain to…

I don't find the utility to outweigh the PITA. I've been on both sides of the fence, and maintained large projects with heavy protobuf use.

I don't find the schema validation powerful enough. You still have to write correct code to migrate semantics. Avoiding incorrect reuse of field names is... nice, but also the most trivial of the problems.

(I do like schemas in theory. It's possible I haven't worked with good enough tooling around protobufs to really experience joy resulting from those schemas. The protos themselves certainly aren't guaranteed to be self-documenting, in some of my experiences.)

I don't find code generation results to be smooth in many languages. At $company, we switched through no less than three different final-stage code generators in search of a smooth experience. Not all of this was misplaced perfectionism: in some cases, it was driven by the sheer absurd method count verbosity in one code generator basically making it impossible to ship an application. (You've perhaps heard of the method count limits in early android? Bam.)

I don't think the debug decoding tools for protobuf are directly comparable to CBOR's interoperability with JSON. CBOR and JSON are literally interchangable. That means I can flip a bit in my program config and start using one instead of the other. Config files? Flip the bit. Network traffic not meant for humans? Flip the bit. Need to debug on the network? Flip the bit. Want to pipe it into other tools like `jq`? No problem. There's a whole ecosystem around JSON, and I like that ecosystem. Working with CBOR, I can have the best of both worlds.

Sometimes opinionated is good and powerful and really makes things streamlined in a way that contributes to the overall utility of the system. I don't think this is one of them. Almost every major feature of gRPC I'm interested in -- multiplexing, deadlines and cancellations, standard definitions for bidi streaming, etc -- has nothing to do with protobufs.

natch, I used "I" heavily in this comment, because I realize these are heavily subjective statements, and not everyone shares these opinions :)

Re: gRPC: Internet-scale RPC framework is now 1.0

#110

Post by wanghq »

hnbroseph wrote: post #80
spraak wrote: post #63

Can anyone explain like I'm 5 what an RPC framework is?

whenever you hit a json api, that's effectively an RPC call. for example, if you visit: https://api.github.com/repos/grpc/grpc/issues you'll get some json back. what happens is the browser resolves "api.github.com" (via DNS) to an IP address, and then opens a socket connection to that address. then, in accordance with the HTTP protocol it executes a "GET" operation (a notion particular to HTTP), and in response the g…

JSON API != RPC

Also the github api is more RESTful than RPC.

Post reply on HN