eBPF will help solve service mesh by getting rid of sidecars
1–10 of 110 posts
Re: eBPF will help solve service mesh by getting rid of sidecars
#2Offtopic: I really like the style of the diagrams. I remember seeing something similar elsewhere. Are this manually drawn or is this the result of some tool?
Re: eBPF will help solve service mesh by getting rid of sidecars
#3Honestly after I learned that the majority of Kubernetes nodes just proxy traffic between each other using iptables and that a load balancer can't tell the nodes apart (ones where your app lives vs ones that will proxy connection to your app) I got really worried about any kind of persistent connection in k8s land.
Since some number of persistent connections will get force terminated on scale down or node replacement events...
Cilium and eBPF looks like a pretty good solution to this though since you can then advertise your pods directly on the network and load balance those instead of every node.
Re: eBPF will help solve service mesh by getting rid of sidecars
#4Offtopic: I really like the style of the diagrams. I remember seeing something similar elsewhere. Are this manually drawn or is this the result of some tool?
OP here: It's whimsical.com. I really love it.
Re: eBPF will help solve service mesh by getting rid of sidecars
#5Re: eBPF will help solve service mesh by getting rid of sidecars
#6Honestly after I learned that the majority of Kubernetes nodes just proxy traffic between each other using iptables and that a load balancer can't tell the nodes apart (ones where your app lives vs ones that will proxy connection to your app) I got really worried about any kind of persistent connection in k8s land. Since some number of persistent connections will get force terminated on scale down or node replacement…
Whether load balancer can or can-not tell the nodes apart depends on load balancer and method you use to expose your service to it, as well as what kind of networking setup you use (i.e. is pod networking sensibly exposed to load balancer or ... weirdly)
Each "Service" object provides (by default, can be disabled) load-balanced IP address that by default uses kube-proxy as you described, a DNS A record pointing to said address, DNS SRV records pointing to actual direct connections (whether NodePorts or PodIP/port combinations) plus API access to get the same data out.
There are even replacement kube-proxy implementations that route everything through F5 load balancer boxes, but they are less known.
Re: eBPF will help solve service mesh by getting rid of sidecars
#7Honestly after I learned that the majority of Kubernetes nodes just proxy traffic between each other using iptables and that a load balancer can't tell the nodes apart (ones where your app lives vs ones that will proxy connection to your app) I got really worried about any kind of persistent connection in k8s land. Since some number of persistent connections will get force terminated on scale down or node replacement…
> Honestly after I learned that the majority of Kubernetes nodes just proxy traffic between each other using iptables and that a load balancer can't tell the nodes apart (ones where your app lives vs ones that will proxy connection to your app) I got really worried about any kind of persistent connection in k8s land.
There can be a difference, if your LoadBalancer-type service integration is well implemented. The externalTrafficPolicy knob determines whether all nodes should attract traffic from outside or only nodes that contain pods backing this service. For example, metallb (which attracts traffic by /32 BGP announcements to given external peers) will do this correctly.
Within the cluster itself, only nodes which have pods backing a given service will be part of the iptables/ipvs/... Pod->Service->Pod mesh, so you won't end up with scenic routes anyway. Same for Pod->Pod networking, as these addresses are already clustered by host node.
Re: eBPF will help solve service mesh by getting rid of sidecars
#8So instead of making the applications use a good RPC library, we're going to shove more crap into the kernel? No thanks, from a security context and complexity perspective.
Per https://blog.dave.tf/post/new-kubernetes/ , the way that this was solved in Borg was:
> "Borg solves that complexity by fiat, decreeing that Thou Shalt Use Our Client Libraries For Everything, so there’s an obvious point at which to plug in arbitrarily fancy service discovery and load-balancing. "
Which seems like a better solution, if requiring some reengineering of apps.
Re: eBPF will help solve service mesh by getting rid of sidecars
#9So instead of making the applications use a good RPC library, we're going to shove more crap into the kernel? No thanks, from a security context and complexity perspective. Per https://blog.dave.tf/post/new-kubernetes/ , the way that this was solved in Borg was: > "Borg solves that complexity by fiat, decreeing that Thou Shalt Use Our Client Libraries For Everything, so there’s an obvious point at which to plug in ar…
It is the technically better solution IMO/IME, too.
But that doesn't work when you're trying to sell enterprises the idea of 'just move your workloads to Kubernetes!'. :)
Re: eBPF will help solve service mesh by getting rid of sidecars
#10So instead of making the applications use a good RPC library, we're going to shove more crap into the kernel? No thanks, from a security context and complexity perspective. Per https://blog.dave.tf/post/new-kubernetes/ , the way that this was solved in Borg was: > "Borg solves that complexity by fiat, decreeing that Thou Shalt Use Our Client Libraries For Everything, so there’s an obvious point at which to plug in ar…
What if a client library does not yet exist for your language?