Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Twirp: A new RPC framework for Go

blog.twitch.tv

Post reply on HN
Print view View original

91–99 of 99 posts

Re: Twirp: A new RPC framework for Go

#91

Post by spenczar5 »

bringtheaction wrote: post #80
spenczar5 wrote: post #12

Hey everyone! I'm the OP and primary author of Twirp. I'm happy to answer any questions and hear feedback. You can also reach me directly, if you like, email is in my profile.

The blog post mentions the following: > The core design of Twirp is language-agnostic and we’re planning to expand into new languages, but our Go implementation is already stable and capable of serving heavy production loads. I was wondering, is Rust on the roadmap?

It's not on the roadmap because I have zero experience in Rust. I'd be very happy to see the community make a Rust generator and could provide guidance on getting the protocol side right, but I wouldn't be able to tell whether the generated client was idiomatic - which is really important.

It'll take someone who is very fluent in Rust and who is motivated to do it, but I'm all for it.

Re: Twirp: A new RPC framework for Go

#92

Post by spenczar5 »

hinoka wrote: post #78
spenczar5 wrote: post #12

Hey everyone! I'm the OP and primary author of Twirp. I'm happy to answer any questions and hear feedback. You can also reach me directly, if you like, email is in my profile.

What do you think about prpc, which is our version of "gRPC for http/1.1" https://godoc.org/go.chromium.org/luci/grpc/prpc

I hadn't even heard of it until yesterday, to be honest. I know very little about it. It looks like it does a good job addressing the HTTP 1.1 concern, for sure, but I don't know whether it addresses the other issues we've had. I'd have to spend a lot more time reading to understand it.

Re: Twirp: A new RPC framework for Go

#93

Post by warent »

daveroberts wrote: post #86

I'm trying to understand the problem this solves. Let's say you have an HTTP API which allows users to update their email address: POST /api/user/:username/update_email But you change the application to require API clients to send the user_id instead of the username. POST /api/user/:user_id/update_email Wouldn't you still need to mandate that all clients are updated regardless of whether you use this tool as an abstr…

Recommend reading on what an RPC is: https://www.geeksforgeeks.org/operating-system-remote-proced...

And protobufs: https://developers.google.com/protocol-buffers/docs/proto3

Example, one benefit is that you're defining your API by using language neutral protobufs which then generate code consistently (including types!) into many languages. Your entire communication procedure can be easily and succinctly described in a single small, human readable file.

Re: Twirp: A new RPC framework for Go

#94

Post by lobster_johnson »

bborud wrote: post #89
lobster_johnson wrote: post #68

Earlier quoted context omitted.

On the other hand, plain URLs with JSON are much easier to work with without writing any code. You can do everything want with curl from the shell, and often an API allows doing almost anything from a browser (Elasticsearch comes to mind). The simplicity of it all comes in handy when you want to do something trivial — load a small piece of data into the server, do some diagnostics, run some ad-hoc queries, etc. — wit…

I think the only RPC mechanism I've been happy with, that required little work and didn't constantly get in the way was Stubby - the precursor to gRPC used inside Google. For a few years inside Google I experienced zero discussions about almost every aspect of RPC. It took a trivial amount of time to implement stuff interfaces, clients and servers in multiple languages and it was trivial to understand the interface o…

How does Stubby compare to gRPC?

I imagine part of the lack of friction around Stubby was that Google was the only consumer, and could maintain client and server bindings/tools for the strict subset of the languages that Google standardized on.

Re: Twirp: A new RPC framework for Go

#95

Post by cam-stitt »

pebers wrote: post #82
cam-stitt wrote: post #60

Earlier quoted context omitted.

When I implemented GRPC on ELB, we used a multiplexer to re-route the GRPC requests. See: https://github.com/soheilhy/cmux/ . The only other issue we had was that ELB's would not let connections live longer than 30 seconds.

You can increase that connection timeout, although there's an upper limit of an hour or so which it will eventually exceed.

I completely forgot this. Makes it a lot more feasible to use it.

Re: Twirp: A new RPC framework for Go

#96

Post by bborud »

lobster_johnson wrote: post #94
bborud wrote: post #89

Earlier quoted context omitted.

I think the only RPC mechanism I've been happy with, that required little work and didn't constantly get in the way was Stubby - the precursor to gRPC used inside Google. For a few years inside Google I experienced zero discussions about almost every aspect of RPC. It took a trivial amount of time to implement stuff interfaces, clients and servers in multiple languages and it was trivial to understand the interface o…

How does Stubby compare to gRPC? I imagine part of the lack of friction around Stubby was that Google was the only consumer, and could maintain client and server bindings/tools for the strict subset of the languages that Google standardized on.

It was pretty similar, but gRPC is a bit simpler since Stubby had a lot of other stuff to deal with authorization etc.

I wouldn't say the lack of friction was mostly due to Google being the only consumer. It was mostly because there was a clear path from A to B when you wanted to give something an RPC interface and that this path was made efficient.

Or at least more efficient than trying to use REST-like stuff in a large organization with lots of different teams using different technologies.

It also helped that it wasn't a democracy. You had to use it. If you didn't like that you were free to leave. As a result people will focus more effort on making the tools better and make friction points go away.

In practical terms: we can spend weeks on getting a REST-like interface to work with other projects because everyone has an opinion on every bit of the design, and everyone uses different, and quirky libraries and tools. For Stubby in Google back then, it was mostly about defining the data structures, the RPC calls, discuss semantics and then the mechanics were taken care of. This is far, far, far from the actual case for many other technologies.

(And while I appreciate HATEOAS as a design philosophy, and I've tried to make use of it several times, it just is not worth the effort. It is just takes too much time to do right and to get everyone on the same page. Most proponents are more keen on telling everyone how they are using it wrong, than on writing good tools that actually help people use it right. There's very little empathy with the developer).

Re: Twirp: A new RPC framework for Go

#97

Post by doh »

spenczar5 wrote: post #88
nickspacek wrote: post #87

Earlier quoted context omitted.

What about allowing/respecting an Accept header in the Request? In @doh's case, if the client only specified Accept: application/protobuf that would override the default behavior of returning JSON encoded errors.

That’s a pretty good idea. It does expand the complexity of the client a bit, but at least it’s in an opt-in way so it doesn’t strictly need to be done for cross-language clients. But what would the benefit(s) to users be? If they are deserializing a protobuf error, they are almost certainly using a generated client, so I don’t think they will know or care how the error was encoded. (This might be better as a github…

I think it depends who the user is in your case. In mine it's the developer who has to work with Twirp outside of the standard libraries (maybe different language, maybe just wants to incorporate it in their own code, ...).

I also like when things are consistent without surprising behavior.

Re: Twirp: A new RPC framework for Go

#98

Post by mikeschinkel »

It is amazing to me that almost nobody here actually questioned the wisdom of throwing out the time tested benefits of robustness of REST in exchange for that which REST was created to eliminate; the fragility of RPC. And all because using RPC is easier in the moment (vs. over time.)

This reminds of of the old saw "Those who ignore history are doomed to repeat it."

If you are unaware of the benefits, here are a just few links that can explain it:

- https://www.quora.com/What-are-the-advantages-of-REST-over-a...

- https://www.quora.com/What-is-the-difference-between-REST-an...

- http://duncan-cragg.org/blog/post/getting-data-rest-dialogue...

- https://apihandyman.io/do-you-really-know-why-you-prefer-res...

- https://www.quora.com/What-are-the-pros-and-cons-of-REST-ver...

Re: Twirp: A new RPC framework for Go

#99

Post by Walkman »

mikeschinkel wrote: post #98

It is amazing to me that almost nobody here actually questioned the wisdom of throwing out the time tested benefits of robustness of REST in exchange for that which REST was created to eliminate; the fragility of RPC. And all because using RPC is easier in the moment (vs. over time.) This reminds of of the old saw "Those who ignore history are doomed to repeat it." If you are unaware of the benefits, here are a just…

I did. It's pretty obvious they don't understand REST at all so they reinvented the wheel.
Post reply on HN