Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

eBPF will help solve service mesh by getting rid of sidecars

isovalent.com

Post reply on HN
Print view View original

51–60 of 110 posts

Re: eBPF will help solve service mesh by getting rid of sidecars

#51

Post by pm90 »

dijit wrote: post #3

Honestly after I learned that the majority of Kubernetes nodes just proxy traffic between each other using iptables and that a load balancer can't tell the nodes apart (ones where your app lives vs ones that will proxy connection to your app) I got really worried about any kind of persistent connection in k8s land. Since some number of persistent connections will get force terminated on scale down or node replacement…

This is a concern only if you have ungraceful node termination Ie you suddenly yoink the node. In most cases when you terminate the node, k8s will (attempt to) cordon and drain the nodes, letting the pods gracefully terminate the connections before getting evicted.

If you didn’t have k8s and just used an autoscaling group of VMs you would have the same issue…

Re: eBPF will help solve service mesh by getting rid of sidecars

#52

Post by jayd16 »

zdw wrote: post #8

So instead of making the applications use a good RPC library, we're going to shove more crap into the kernel? No thanks, from a security context and complexity perspective. Per https://blog.dave.tf/post/new-kubernetes/ , the way that this was solved in Borg was: > "Borg solves that complexity by fiat, decreeing that Thou Shalt Use Our Client Libraries For Everything, so there’s an obvious point at which to plug in ar…

It does feel a bit like we're trying to monkey patch compiled code but the benefits are pretty clear.

Re: eBPF will help solve service mesh by getting rid of sidecars

#53

Post by allset_ »

tgraf wrote: post #45
codetrotter wrote: post #15

> Identity-based Security: Relying on network identifiers to achieve security is no longer sufficient, both the sending and receiving services must be able to authenticate each other based on identities instead of a network identifier. Kinda semi-offtopic but I am curious to know if anyone has used identity part of a WireGuard setup for this purpose. So say you have a bunch of machines all connected in a WireGuard VP…

One of the methods that Cilium (which implements this eBPF-based service mesh idea) uses to implementation authentication between workloads is Wireguard. It does exactly what you describe above. In addition it can also be used to enforce based on service specific keys/certificates as well.

Isn't the Wireguard implementation in Cilium between nodes only, not workloads (pods)?

Re: eBPF will help solve service mesh by getting rid of sidecars

#55

Post by nonameiguess »

zdw wrote: post #8

So instead of making the applications use a good RPC library, we're going to shove more crap into the kernel? No thanks, from a security context and complexity perspective. Per https://blog.dave.tf/post/new-kubernetes/ , the way that this was solved in Borg was: > "Borg solves that complexity by fiat, decreeing that Thou Shalt Use Our Client Libraries For Everything, so there’s an obvious point at which to plug in ar…

In addition to whether or not all of your various dev teams preferred languages have a supported client SDK, you also have the build vs. buy issue if you're plugging COTS applications into your service mesh, there is no way to force a third party vendor to reengineer their application specifically for you.

This probably dictates a lot of Google's famous "not invented here" behavior, but most organizations can't afford to just write their entire toolchain from scratch and need to use applications developed by third parties.

Re: eBPF will help solve service mesh by getting rid of sidecars

#56

Post by darkwater »

cfors wrote: post #49
darkwater wrote: post #44

Earlier quoted context omitted.

> Especially if you have specific sidecars that only work on a specific infrastructure, for example if you have a Vault sidecar that deals with secrets for your service over EKS IAM permissions, you suddenly can't start your service without a decent amount of mocking and feature flags. Its nice to not have to burden your client code with all of that. Could you please elaborate on this? I don't fully understand what y…

Take vault for example. Rather than have to toggle a flag in your service to get a secret, you could have the vault sidecar inject the secret automatically into your container, as opposed to having to pass a configuration flag `USE_VAULT` to your application, which will conditionally have a baked in vault client that fetches your secret for you. Your service doesn't really care where the secret comes from, as long it…

Ok, so you are indeed advocating for the sidecar approach (and on this I fully agree, especially this Vault example)

Re: eBPF will help solve service mesh by getting rid of sidecars

#57

Post by topspin »

dharmab wrote: post #31
dijit wrote: post #3

Honestly after I learned that the majority of Kubernetes nodes just proxy traffic between each other using iptables and that a load balancer can't tell the nodes apart (ones where your app lives vs ones that will proxy connection to your app) I got really worried about any kind of persistent connection in k8s land. Since some number of persistent connections will get force terminated on scale down or node replacement…

That's if you're using a NodePort service, which the documentation explains is for niche use cases such as if you don't have a compatible dedicated load balancer. In most professional setups you do have such a load balancer and can use other types of routing that avoid this. https://kubernetes.io/docs/concepts/services-networking/serv...

> In most professional setups you do have such a load balancer

May I ask what one might use in an AWS cloud environment to provide that load balancer within a Region?

Does IPv6 address any of these issues? It seems to me that IPv6 is capable of providing every component in the system its own globally routable address, identity (mTLS perhaps) and transparent encryption with no extra sidecars, eBPF pieces, etc.

Re: eBPF will help solve service mesh by getting rid of sidecars

#58

Post by tgraf »

allset_ wrote: post #53
tgraf wrote: post #45

Earlier quoted context omitted.

One of the methods that Cilium (which implements this eBPF-based service mesh idea) uses to implementation authentication between workloads is Wireguard. It does exactly what you describe above. In addition it can also be used to enforce based on service specific keys/certificates as well.

Isn't the Wireguard implementation in Cilium between nodes only, not workloads (pods)?

It can do both. It can authenticate and encrypt all traffic between nodes which then also encrypts all traffic between the pods running on those pods. This is great because it also covers pod to node and all control plane traffic. The encryption can also use specific keys for different services to authenticate and encrypt pod to pod individually.

Re: eBPF will help solve service mesh by getting rid of sidecars

#59

Post by starfallg »

madjam002 wrote: post #23
codetrotter wrote: post #15

> Identity-based Security: Relying on network identifiers to achieve security is no longer sufficient, both the sending and receiving services must be able to authenticate each other based on identities instead of a network identifier. Kinda semi-offtopic but I am curious to know if anyone has used identity part of a WireGuard setup for this purpose. So say you have a bunch of machines all connected in a WireGuard VP…

I too am interested in this. I long for the day where Kubernetes services, virtual machines, dedicated servers and developer machines can all securely talk to eachother in some kind of service mesh, where security and firewalls can be implemented with "tags". Tailscale seems to be pretty much this, but while it seems great for the dev/user facing side of things (developer machine connectivity), it doesn't seem like i…

>I long for the day where Kubernetes services, virtual machines, dedicated servers and developer machines can all securely talk to eachother in some kind of service mesh, where security and firewalls can be implemented with "tags".

That's exactly what Scalable Group Tags (SGTs) are -

https://tools.ietf.org/id/draft-smith-kandula-sxp-07.html

Cisco implements this as a part of TrustSec

Re: eBPF will help solve service mesh by getting rid of sidecars

#60

Post by shosti »

topspin wrote: post #57
dharmab wrote: post #31

Earlier quoted context omitted.

That's if you're using a NodePort service, which the documentation explains is for niche use cases such as if you don't have a compatible dedicated load balancer. In most professional setups you do have such a load balancer and can use other types of routing that avoid this. https://kubernetes.io/docs/concepts/services-networking/serv...

> In most professional setups you do have such a load balancer May I ask what one might use in an AWS cloud environment to provide that load balancer within a Region? Does IPv6 address any of these issues? It seems to me that IPv6 is capable of providing every component in the system its own globally routable address, identity (mTLS perhaps) and transparent encryption with no extra sidecars, eBPF pieces, etc.

Ingresses on EKS will set up an ALB that sends traffic directly to pods instead of nodes (basically skips the whole K8s Service/NodePort networking setup). You have to use ` alb.ingress.kubernetes.io/target-type: ip` as an annotation I think (see https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress...).
Post reply on HN