Any performance numbers?
Twirp: A new RPC framework for Go
31–40 of 99 posts
Re: Twirp: A new RPC framework for Go
#32This is a pretty awesome project. The one thing that's missing would be autogenerated javascript/typescript stubs like grpc-web does. Will definitely experiment with this when building small go applications.
Hopefully these will be implemented soon by the community. Open source rules :)
Re: Twirp: A new RPC framework for Go
#33This looks promising! We use the go-grpc SDK in conjunction with gogoprotobuf, and it's been a rocky road. While the article identifies some operational issues (e.g. the reliance on HTTP/2), there are several considerable deficiencies with gRPC today, at least when using it with Go: 1. The JSON mapping (jsonpb.go) is clumsy at best, and by this I mean that it produces JSON that often doesn't look anything like how yo…
Yeah, I agree with pretty much everything you've written here.
> 1. The JSON mapping (jsonpb.go) is clumsy at best
The best thing for optional fields in jsonpb is to use the protobuf wrapper types [1]. They have special support in jsonpb to serialize and deserialize as you would expect, without the indirection. But the Go structs you get on the other end are a little weird, so its a tradeoff.
> 2. The Go code generator usually produces highly unidiomatic Go.
Yeah, using the generated structs as the main domain types in your code can be up-and-down. I agree that gogoprotobuf can help, but it's rough. We definitely use Getter methods on generated structs quite a bit for stuff like oneofs.
> 3. Proto3 has very limited support for expressing "free-form" data.
There's always `repeated byte` :) It sounds like a joke, but we've used it in some spots where the input is totally schema-less.
The Any type is also designed for this sort of thing. Still clumsy, though.
[1] https://github.com/google/protobuf/blob/master/src/google/pr...
Re: Twirp: A new RPC framework for Go
#34Any performance numbers?
It's really hard to write benchmarks of an RPC system that mean much, but the overhead is really just in serialization. We have services that handle tens of thousands of requests per second on Twirp in one process.
Serialization/deserialization of a typical protobuf struct takes a microsecond or two, but it generates some garbage, so GC ends up slowing you down if you try to go really crazy and push past 100k req/s in one process with non-trivial message structures.
Re: Twirp: A new RPC framework for Go
#35What could make this really take off is an in-browser JS client. The simplicity it has added seems to really help there. The gRPC team has had one in hiding for a long time only giving people access who explicitly ask: https://github.com/grpc/grpc/issues/8682 (good thing GitHub has a feature that snips hundreds of comments or that link would take a while to load)
Re: Twirp: A new RPC framework for Go
#36Why use HTTP for transport instead of Messagepack or ZMQ? Seems a bit overkill if you are whipping binary data back and forth between services. Protobuf + ZMQ seems a lot more efficient to me.
Re: Twirp: A new RPC framework for Go
#37What could make this really take off is an in-browser JS client. The simplicity it has added seems to really help there. The gRPC team has had one in hiding for a long time only giving people access who explicitly ask: https://github.com/grpc/grpc/issues/8682 (good thing GitHub has a feature that snips hundreds of comments or that link would take a while to load)
Totally agree, and it's something I'd love to see. Consider this a call for contributors - I think a simple generated javascript client would be an excellent way to help with the project.
Re: Twirp: A new RPC framework for Go
#38It is nice but would say with limited value outside of Twitch
Re: Twirp: A new RPC framework for Go
#39Any performance numbers?
It's really hard to write benchmarks of an RPC system that mean much, but the overhead is really just in serialization. We have services that handle tens of thousands of requests per second on Twirp in one process. Serialization/deserialization of a typical protobuf struct takes a microsecond or two, but it generates some garbage, so GC ends up slowing you down if you try to go really crazy and push past 100k req/s i…
Thanks, that's the exact info I was looking for.
I've unfortunately been bitten before by choosing json as a serialization format, specifically in go, due to JSON performance dominating processing.
No criticism though, JSON is the right choice for many types of APIs.
Re: Twirp: A new RPC framework for Go
#40Why use HTTP for transport instead of Messagepack or ZMQ? Seems a bit overkill if you are whipping binary data back and forth between services. Protobuf + ZMQ seems a lot more efficient to me.
A lot of this boils down to wanting to be able to use standard load balancers.