Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linkerd: Twitter-Style Operability for Microservices

blog.buoyant.io

Post reply on HN
Print view View original

31–39 of 39 posts

Re: Linkerd: Twitter-Style Operability for Microservices

#31

Post by olix0r »

brown9-2 wrote: post #11

The "sidecar" proxy model reminds me a lot of https://github.com/airbnb/synapse

We definitely took some inspiration from projects like synapse.

However, Finagle, the core tech behind linkerd, provides some extremely powerful tools to do things like:

- per-request routing to support things like "when I browse the site, use the staging version of the users service and production versions of all other services". (https://twitter.github.io/finagle/guide/Names.html#interpret...)

- request cancellation, so that when a user request timeouts downstream work can be reclaimed

- budget-based timeout management (https://twitter.github.io/finagle/guide/Servers.html#request...)

- circuit breaking (https://twitter.github.io/finagle/guide/Clients.html#failure...)

- etc, etc.

We think that offering these sorts of features in a sidecar model will be extremely powerful.

Re: Linkerd: Twitter-Style Operability for Microservices

#32

Post by jeromyevans »

yeukhon wrote: post #3

Suppose I designed something from scratch at former company, and then I decided to reimplement the same project after leaving the company (and perhaps turn that into an open source project), with mostly new code, but similar concept, would that be considered copyright infringement?

Your employment contract or a related document you've executed may include a post-employment restraint/non-compete. These vary a lot by level of the position and type of work. They are difficult/expensive to enforce and are open to interpretation about "reasonableness". These are generally the former employer's best means to prevent you from immediately reproducing something. However they also can't prevent you from working in your profession and using what you've learned as a professional.

Re: Linkerd: Twitter-Style Operability for Microservices

#33

Post by mavelikara »

njharman wrote: post #19
brown9-2 wrote: post #12

Earlier quoted context omitted.

are there a lot of companies in the practice of copyrighting their internal architecture and patterns?

In usa everything that is copyrightable, becomes copyrighted at moment it is created. For instance that sentence, and this on, are protected works owned by me. The only reAson HN can legally use then is cause I gave them license, buried somewhere in Eula/site terms of use. Also, an architecture is not copyrightable. Maybe patentable. Copyrights protect specific expressions, not a general idea, plan, or architecture.…

But Twitter has a patent pledge with its engineers [1]:

    The IPA is a new way to do patent assignment that 
    keeps control in the hands of engineers and designers. 
    It is a commitment from Twitter to our employees that 
    patents can only be used for defensive purposes. We will
    not use the patents from employees’ inventions in 
    offensive litigation without their permission. What’s
    more, this control flows with the patents, so if we sold
    them to others, they could only use them as the inventor 
    intended.
[1]: https://blog.twitter.com/2012/introducing-the-innovator-s-pa...

Re: Linkerd: Twitter-Style Operability for Microservices

#34

Post by williamallthing »

lobster_johnson wrote: post #28
williamallthing wrote: post #24

Earlier quoted context omitted.

We can definitely improve the docs a bit on this point. But FWIW we totally agree with you. Finagle itself is a JVM library, and that worked well enough at Twitter [insert caveats here], but a big part of the reason we built linkerd is extending that model to non-JVM / polyglot services. There's SO much good stuff in Finagle... it would be a pity to confine it just to the JVM.

Yup. I do wish there were more work done on the actual client/server side (the P and C of "RPC"), too, something Linkerd can't solve for you, and which I believe Finagle does. So far all our work has been JSON-over-HTTP, which is not performant. We've looked at gRPC and Thrift, but we've been dreading the prospect of having to pre-declare IDL, generate language glue and write client/server setup glue for every app. A…

I'm curious to hear what your experience with NATS will be. We haven't played with it. I do know that a ton of the work that went into Finagle was around resilience at scale: things like backpressure, circuit breaking, and generally tolerating slow, bad, or flapping servers. Personally I would want to see that sort of thing in any RPC system before using it at scale.

Re: Linkerd: Twitter-Style Operability for Microservices

#35

Post by lobster_johnson »

williamallthing wrote: post #34
lobster_johnson wrote: post #28

Earlier quoted context omitted.

Yup. I do wish there were more work done on the actual client/server side (the P and C of "RPC"), too, something Linkerd can't solve for you, and which I believe Finagle does. So far all our work has been JSON-over-HTTP, which is not performant. We've looked at gRPC and Thrift, but we've been dreading the prospect of having to pre-declare IDL, generate language glue and write client/server setup glue for every app. A…

I'm curious to hear what your experience with NATS will be. We haven't played with it. I do know that a ton of the work that went into Finagle was around resilience at scale: things like backpressure, circuit breaking, and generally tolerating slow, bad, or flapping servers. Personally I would want to see that sort of thing in any RPC system before using it at scale.

We're a small shop; each of our clusters are currently a couple dozen static servers at most. So we don't really do anything at what would be considered "at scale". For us, developer ease of use and performance is a higher priority.

But we're slowly moving towards a setup where we'll likely rely on autoscaling, and where we will likely need a more robust architecture.

Re: Linkerd: Twitter-Style Operability for Microservices

#36

Post by sayrer »

williamallthing wrote: post #25
lobster_johnson wrote: post #21

Looks like the support protocols right now are HTTP, Thrift (framed transport) and something called Mux. Is this intended to be pluggable?

Yup, absolutely. Any requests? :)

GRPC. It's really the only practical choice if you have Go, Node.js, and PHP apps that need to consume one central service. It also has a lot of the benefits that Mux brings.

Re: Linkerd: Twitter-Style Operability for Microservices

#37

Post by olix0r »

sayrer wrote: post #36
williamallthing wrote: post #25

Earlier quoted context omitted.

Yup, absolutely. Any requests? :)

GRPC. It's really the only practical choice if you have Go, Node.js, and PHP apps that need to consume one central service. It also has a lot of the benefits that Mux brings.

Luckily GRPC can downgrade to HTTP/1.1, so if you want some of linkerd's features it isn't a total non-starter; but this is obviously less-than-desirable. We're working with the finagle team to complete finagle's netty4 integration, which will enable us to transparently introduce netty's http/2 codec into linkerd. This is a high-priority feature.

Re: Linkerd: Twitter-Style Operability for Microservices

#38

Post by eikenberry »

This is a good step forward for Finagle as it eliminates the anti-pattern of encapsulating the communications functionality into a library. This inevitably turns any collection of services into a distributed monolith, killing the loose coupling that is the point.

Re: Linkerd: Twitter-Style Operability for Microservices

#39

Post by sayrer »

olix0r wrote: post #37
sayrer wrote: post #36

Earlier quoted context omitted.

GRPC. It's really the only practical choice if you have Go, Node.js, and PHP apps that need to consume one central service. It also has a lot of the benefits that Mux brings.

Luckily GRPC can downgrade to HTTP/1.1, so if you want some of linkerd's features it isn't a total non-starter; but this is obviously less-than-desirable. We're working with the finagle team to complete finagle's netty4 integration, which will enable us to transparently introduce netty's http/2 codec into linkerd. This is a high-priority feature.

that's a fair answer.

It seems like there's overlap beyond lack of HTTP/2 support, where some of Finagle's features are handled by GRPC in different ways. I'd quite like to use the GRPC way, since Google is maintaining the bindings to many languages, and doing a decent job.

Post reply on HN