Connecting Kubernetes services with linkerd
1–10 of 49 posts
Re: Connecting Kubernetes services with linkerd
#2Great article, I've been able to learn more about why a cluster setup could utilize linkerd. However this statement confused me a little:
However, Finagle is written in Scala, which Gould concedes is "not for everyone".
[1] https://github.com/linkerd/linkerd [2] https://github.com/linkerd/linkerd-tcp
Re: Connecting Kubernetes services with linkerd
#3Great article, I've been able to learn more about why a cluster setup could utilize linkerd. However this statement confused me a little: However, Finagle is written in Scala, which Gould concedes is "not for everyone". linkerd is written in Scala [1] and linkerd is written in Rust [2]. If he is optimizing for other people’s contributions, is this really the best way to go? [1] https://github.com/linkerd/linkerd [2]…
Maybe it means at runtime, as opposed to at contribution-time?
Re: Connecting Kubernetes services with linkerd
#4I appreciate the sentiment behind linkerd et al but I think we are really over-complicating things and tying ourselves into knots by going head-first into this world. I can't help but think we're going to regret it in a few years.
The complexity introduced not only by Docker, but also k8s and all of the other things competing for a spot in the middle of your cluster, is immense and widely underestimated.
Your applications need to be aware and adapted to the k8s model of automatic pod termination, no hard storage, etc. Google has had over 10 years with the best computer scientists money can buy to perfect their stack. Most companies don't.
For most companies, k8s/linkerd/Docker deployments are a case of overengineering run frighteningly amok.
There are many-a-snafu lying down this path. I'm very worried about how casually everyone is willing to jump aboard.
One particular worry: most Dockerfiles are built on opaque images registered with an image registry, and many of the publishers are not well-known. If you want Ruby, the habit is to go find someone's "pre-configured" alpine-ruby image (for instance). These are frequently distributed by complete strangers. Building "from scratch" off the base image distributed by the distro itself is considered "the old way", at least among those I've worked with.
My understanding is that most public image repositories are unmoderated. The moment a widely-used image gets pwned, there is going to be panic.
Source: been converting entire company to Docker/k8s
Re: Connecting Kubernetes services with linkerd
#5> Facilities that linkerd provides to assist with this include service discovery, load balancing, encryption, tracing and logging, handling retries, expiration and timeouts, back-offs, dynamic routing, and metrics.
Why not use SRV records and TLS? Retries, expiration, timeouts, and back-offs can all be placed into the networking library, and are, commonly.
Also, regarding https://linkerd.io/, what's with landing pages having almost no useful information on them. "linker∙d is a transparent proxy that adds service discovery, routing, failure handling, and visibility to modern software applications" what's that even mean?
> Fast, lightweight, and performant > Handles tens of thousands of requests per second per instance with minimal latency overhead. Scales horizontally with ease.
Well, since you're calling yourself "transparent" I would hope it'd be fast!
> Any language, any environment > Runs as a transparent proxy alongside existing applications, integrates with existing infrastructure.
So, it's just proxying and retrying things and doing tls for me? Why isn't this better handled in the library I'm using to make the call in the first place?
> Latency-aware load-balancing > Balances request traffic using real-time performance, reducing tail latencies across your application.
Sounds like it requires a lot of cross-talk on my network, eating up bandwidth. That doesn't sound lightweight.
> Runtime traffic routing > Provides dynamic, scoped, logical routing rules, enabling blue-green deployments, staging, canarying, failover, and more.
OK, some of that does sound useful, but some of it still feels like something the caller should handle.
> Drop-in service discovery > Integrates with most service discovery systems, decoupling applications from specific implementations.
What's this mean? Why not just use SRV records?
> Production-tested and proven at scale > Powers the production infrastructure of banks, artificial intelligence companies, social networks, government labs, and more.
I know why this is here, but it feels like _everyone_ claims the same thing, so, to me, it just becomes noise.
> "linkerd allows you to drop in a transparent layer of application resilience and provides the operational affordances critical for modern, cloud native environments." > —Oliver Gould, CTO, Buoyant
This sounds doesn't make sense to me, you can't "drop in" resiliency. It needs to be built. The whole thing just feels like it's all marketing and doesn't tell me exactly what problems it's solving for me as a developer.
------------
Maybe I'm just an old fogie, or being dense, but I still don't see how this solves any problems I can't and aren't already solving.
Re: Connecting Kubernetes services with linkerd
#6I appreciate the sentiment behind linkerd et al but I think we are really over-complicating things and tying ourselves into knots by going head-first into this world. I can't help but think we're going to regret it in a few years. The complexity introduced not only by Docker, but also k8s and all of the other things competing for a spot in the middle of your cluster, is immense and widely underestimated. Your applica…
> Building "from scratch" off the base image distributed by the distro itself is considered "the old way"
Then I don't want to be new. I started using a few of the Docker Hub images (mostly `openjdk`), but I've transitioned completely to building all runtime images off of `debian:jessie`. And even that image I may want to remove eventually, just so that our CI system isn't affected by Docker Hub outages.
Re: Connecting Kubernetes services with linkerd
#7I appreciate the sentiment behind linkerd et al but I think we are really over-complicating things and tying ourselves into knots by going head-first into this world. I can't help but think we're going to regret it in a few years. The complexity introduced not only by Docker, but also k8s and all of the other things competing for a spot in the middle of your cluster, is immense and widely underestimated. Your applica…
Yes, it's insane. I can only attribute it to that I don't work on problem of any scale (amount of apps, servers, deployments per day, etc.) and all of these people do. Who really wants to spend their time and effort running a Kubernetes master and nodes, 3+ Consul/etcd nodes, and now some fancy load balancers? The ratio of companies that would benefit from that to the number of companies working on products in the space is way out of wack.
Re: Connecting Kubernetes services with linkerd
#8Great article, I've been able to learn more about why a cluster setup could utilize linkerd. However this statement confused me a little: However, Finagle is written in Scala, which Gould concedes is "not for everyone". linkerd is written in Scala [1] and linkerd is written in Rust [2]. If he is optimizing for other people’s contributions, is this really the best way to go? [1] https://github.com/linkerd/linkerd [2]…
> linkerd is written in Scala [1] and linkerd is written in Rust [2]
Though your links are correct your statement isn't. Linkerd is written in Scala, but the Rust bit is linkerd-tcp, a TCP loadbalancer, which linkerd (the Scala one) doesn't handle as that covers HTTP and gRPC with optionally encrypting connections between linkers using TLS.
> If he is optimizing for other people’s contributions, is this really the best way to go?
But is he optimising for other people's contributions? Using a C/Rust for something like a TCP proxy makes a lot more sense to me than having a JVM do that. And if he perceives Scala as "not for everyone", maybe adding Rust means he might get more contributions to those components?
I think that statement actually refers to Finagle itself being written in Scala, so if you want to use it you have to at least hoist your own apps on the JVM and integrate Finagle in them. Finagle is essentially client-side and requires integration into your app tightly coupling the two. By spinning it out as a standalone component that acts as a proxy, like linkerd is doing, Scala doesn't get in the way and what Finagle offers can be used with a mesh of microservices written in a myriad of languages.
Re: Connecting Kubernetes services with linkerd
#9> Facilities that linkerd provides to assist with this include service discovery, load balancing, encryption, tracing and logging, handling retries, expiration and timeouts, back-offs, dynamic routing, and metrics. Why not use SRV records and TLS? Retries, expiration, timeouts, and back-offs can all be placed into the networking library, and are, commonly. Also, regarding https://linkerd.io/ , what's with landing pag…
Disclaimer: I'm still researching linkerd and haven't tried to use it.
One component we're interested in is its "automatic" tracing. It claims to be able to annotate requests with Zipkin stats without requiring any instrumentation at the software level.
You are correct that other solutions exist for service discovery, but that's an in-demand feature right now as well, and afaik the main reason that people use linkerd.
>Maybe I'm just an old fogie, or being dense, but I still don't see how this solves any problems I can't and aren't already solving.
Quick, turn back now from this whole containerization thing, because I assure you this is basically the answer for everything in this space. And you will be aggressively accused of being an old fogie for resisting it. Simplicity is just not appreciated anymore; at this point, "containers" may as well be an antonym for it (and by extension, an antonym for "sane engineering").
Re: Connecting Kubernetes services with linkerd
#10> Facilities that linkerd provides to assist with this include service discovery, load balancing, encryption, tracing and logging, handling retries, expiration and timeouts, back-offs, dynamic routing, and metrics. Why not use SRV records and TLS? Retries, expiration, timeouts, and back-offs can all be placed into the networking library, and are, commonly. Also, regarding https://linkerd.io/ , what's with landing pag…
> > Latency-aware load-balancing > Balances request traffic using real-time performance, reducing tail latencies across your application.
> Sounds like it requires a lot of cross-talk on my network, eating up bandwidth. That doesn't sound lightweight.
Why does that require eating bandwidth? It's a proxy. It can derive statistics from past and current connections and make decisions based on that data for new ones without talking to anyone else.
> Maybe I'm just an old fogie, or being dense, but I still don't see how this solves any problems I can't and aren't already solving.
So? If you're happy with your solution, keep it! But maybe others haven't solved all these problems, or don't like your approach to how you've solved them for other reasons or don't even want to spend time solving these problems in the first place. For many a component like this means it's 10 less things to deal with and more time to focus on building features that'll matter to their end users.
Besides, just because it can already be solved doesn't mean it's not worth trying out other approaches.