61–70 of 99 posts
Earlier quoted context omitted.
gRPC connections are persistent, so a single client will always talk to a single backend in that configuration. It's fine if you have lots of clients but can easily lead to load imbalances. That's why projects such as Envoy exist. I'd link it, but I'm on mobile. Keep an eye on it.
You can use round robin load balancing in gRPC without Envoy.
It is nice but would say with limited value outside of Twitch
I definitely would prefer to use this instead of bizare-REST-like (it's how REST usually devolves into) in a next project , if I can't use graphql.
There seems to be somewhat of a pattern of Go being linked to outages (CloudFlare and now Twitch). Any regrets investing in Go?
The CloudFlare outage was related to leap second handling... while the particulars of the Go library contributed, this is also far, far from the only time that a leap second caused havoc online. Hell, in 2008, Zunes were crippled by a leap day bug.
RPC and time handling are notoriously tricky problems to get right.
There seems to be somewhat of a pattern of Go being linked to outages (CloudFlare and now Twitch). Any regrets investing in Go?
This looks promising! We use the go-grpc SDK in conjunction with gogoprotobuf, and it's been a rocky road. While the article identifies some operational issues (e.g. the reliance on HTTP/2), there are several considerable deficiencies with gRPC today, at least when using it with Go: 1. The JSON mapping (jsonpb.go) is clumsy at best, and by this I mean that it produces JSON that often doesn't look anything like how yo…
Yeah, I agree with pretty much everything you've written here. > 1. The JSON mapping (jsonpb.go) is clumsy at best The best thing for optional fields in jsonpb is to use the protobuf wrapper types [1]. They have special support in jsonpb to serialize and deserialize as you would expect, without the indirection. But the Go structs you get on the other end are a little weird, so its a tradeoff. > 2. The Go code generat…
> using the generated structs as the main domain types in your code can be up-and-down
At $DAYJOB we solve this by doing code generation outward from our domain types. The RPC layer is idiomatic Go because that's what we began with.
Some go/token and regexes take our structs and produce a server-side router implementation for net/http (endpoints from magic comments), some client-side libraries for Go / C++ (Qt) / PHP / JS, and documentation in markdown.
Our system is in a pretty reusable state, but nobody has the free cycles to open it. If Twirp had been available 24mo ago our project might have been different.
Really excited about this. I didn't like how opaque and heavy gRPC is. Also I really wanted support for JSON. Mostly for these reasons, RCP hasn't been implemented in my architecture yet (Just using standard REST) Twirp is everything I wanted in an RPC framework and I'm looking forward to implementing it ASAP. Thanks Twitch team :)
Can I ask why you want JSON with gRPC? The benefits to protobuf are tremendous, with little to no downsides
Debugging with lower-level tools like strace and tcpdump is also something that's trivial with JSON-over-HTTP/1.1, but virtually impossible with gRPC. (I mean, you could grab the payload and run it through gRPC deserialization, but would you?)
I'm a big fan of gRPC, but it is pretty top-heavy — lots of code to accomplish relatively little. If you have a language that can build client bindings dynamically from .proto files without recompilation, that would ease things a lot, but if you're using Go, for example, the bar is pretty high going from zero to productive.
Earlier quoted context omitted.
Can I ask why you want JSON with gRPC? The benefits to protobuf are tremendous, with little to no downsides
On the other hand, plain URLs with JSON are much easier to work with without writing any code. You can do everything want with curl from the shell, and often an API allows doing almost anything from a browser (Elasticsearch comes to mind). The simplicity of it all comes in handy when you want to do something trivial — load a small piece of data into the server, do some diagnostics, run some ad-hoc queries, etc. — wit…
Hey everyone! I'm the OP and primary author of Twirp. I'm happy to answer any questions and hear feedback. You can also reach me directly, if you like, email is in my profile.
> If there is an error, it will be JSON encoded, including a message and a standardized error code.
Why not to return standard protobuf error when the source was a protobuf? It massively complicates things when you have to expect errors in one format and responses in the other.