Looks like the support protocols right now are HTTP, Thrift (framed transport) and something called Mux. Is this intended to be pluggable?
Linkerd: Twitter-Style Operability for Microservices
21–30 of 39 posts
Re: Linkerd: Twitter-Style Operability for Microservices
#22Totally unrelated: the effect/animation when you hover over the avatars (Safari) at https://buoyant.io/#team is really weird. Not sure if it's intentional.
This was not intentional. Thanks for pointing this out.
Re: Linkerd: Twitter-Style Operability for Microservices
#23This is actually closer to Google-Style operability than it is to Twitter-Style operability. :) Twitter doesn't have an equivalent of GSLB (software load balancer), which is essentially what this is.
Have you seen Google's GSLB? It's not a proxy and it's a rather complex and powerful system. It aggregates and coordinates traffic flow but doesn't directly do any kind of forwarding. You could maybe think of Linkerd like GFE but even still, GFE acts as an edge gateway rather than internal RPC system. This is like a wrapper around stubby or grpc.
Is GSLB different from Google Seesaw, the load balancer that was open-sourced recently?
Re: Linkerd: Twitter-Style Operability for Microservices
#24Earlier quoted context omitted.
No no, you're missing the point, it's an RPC proxy which is language agnostic. It sits along side your application as opposed to being a library that you bake in.
I see. My fault for going straight to the Github repo rather than to linkerd.io.
We can definitely improve the docs a bit on this point.
But FWIW we totally agree with you. Finagle itself is a JVM library, and that worked well enough at Twitter [insert caveats here], but a big part of the reason we built linkerd is extending that model to non-JVM / polyglot services. There's SO much good stuff in Finagle... it would be a pity to confine it just to the JVM.
Re: Linkerd: Twitter-Style Operability for Microservices
#25Looks like the support protocols right now are HTTP, Thrift (framed transport) and something called Mux. Is this intended to be pluggable?
Yup, absolutely. Any requests? :)
Re: Linkerd: Twitter-Style Operability for Microservices
#26"It can be done, but it takes years of thought and work to make everything work well in practice."
Really wish they expressed the cost in man-years. It's a few calendar years, and thousands of man-years.
Re: Linkerd: Twitter-Style Operability for Microservices
#27Looks like the support protocols right now are HTTP, Thrift (framed transport) and something called Mux. Is this intended to be pluggable?
Yup, absolutely. Any requests? :)
Cool! Is HTTP/2.0 already there, or planned? I think HTTP/2.0 is important to get proper pipelining of asynchronous calls.
Re: Linkerd: Twitter-Style Operability for Microservices
#28Earlier quoted context omitted.
I see. My fault for going straight to the Github repo rather than to linkerd.io.
We can definitely improve the docs a bit on this point. But FWIW we totally agree with you. Finagle itself is a JVM library, and that worked well enough at Twitter [insert caveats here], but a big part of the reason we built linkerd is extending that model to non-JVM / polyglot services. There's SO much good stuff in Finagle... it would be a pity to confine it just to the JVM.
Yup. I do wish there were more work done on the actual client/server side (the P and C of "RPC"), too, something Linkerd can't solve for you, and which I believe Finagle does.
So far all our work has been JSON-over-HTTP, which is not performant. We've looked at gRPC and Thrift, but we've been dreading the prospect of having to pre-declare IDL, generate language glue and write client/server setup glue for every app.
At the moment, we're closing in on using NATS as a routing proxy, fulfilling a role very similar to Linkerd. Turns out it's fast enough that you can do RPC with it, and it seems very reliable. Language bindings are good to the point where you have to write almost no glue code. You have to pick a serialization format, and I think Msgpack might work well here.
Re: Linkerd: Twitter-Style Operability for Microservices
#29Earlier quoted context omitted.
Have you seen Google's GSLB? It's not a proxy and it's a rather complex and powerful system. It aggregates and coordinates traffic flow but doesn't directly do any kind of forwarding. You could maybe think of Linkerd like GFE but even still, GFE acts as an edge gateway rather than internal RPC system. This is like a wrapper around stubby or grpc.
Is GSLB different from Google Seesaw, the load balancer that was open-sourced recently?
Yea, it's a completely different use case. Internally Google has something called Maglev which looks a little more like Seesaw but even still its quite different.
I haven't actually seen anything like GSLB anywhere else. With classic load balancing we've always been taught to proxy traffic through a single point of entry. GSLB flips that on its head and rather provides routing information which clients can then use for the next set of requests. I'm attempting to develop something similar within https://github.com/micro but it's going to be a long time before I even get marginally close to something as powerful as Google has.
Re: Linkerd: Twitter-Style Operability for Microservices
#30Earlier quoted context omitted.
Yup, absolutely. Any requests? :)
Cool! Is HTTP/2.0 already there, or planned? I think HTTP/2.0 is important to get proper pipelining of asynchronous calls.
We agree 100% that multiplexing is vital.
HTTP/2 is absolutely on the roadmap. We're actively working with some of the folks at Twitter to get it integrated and tested before introducing it into linkerd.
That said, linkerd is able to provision multiple downstream connections to multiplex requests to other services; but we need to extend this to the serverside to get the best application integration.
You may also be interested in Twitter's mux protocol (https://twitter.github.io/finagle/guide/Protocols.html#mux), which provides this featureset internally at Twitter, primarily for Thrift.