I wonder why they didn't use rendezvous hashing (aka HRW)[0]?
It feels like it would solve all the requirement that they laid out, is fully client side, and doesn't require real time updates for the host list via discovery.
Intelligent Kubernetes Load Balancing at Databricks
11–20 of 30 posts
Re: Intelligent Kubernetes Load Balancing at Databricks
#12Less featureful than this, but we’ve been doing GRPC client side load balancing with kuberesolver[1] since 2018. It allows GRPC to handle the balancer implementations. It’s been rock solid for more than half a decade now. 1: https://github.com/sercand/kuberesolver
kuberesolver is an interesting take as well. Directly watching the K8s API from each client could raise scaling concerns at very large scale, but it does open the door to using richer Kubernetes metadata for smarter load-balancing decisions. thanks for sharing!
I think with some rate limiting, it can scale. But it might be a security issue as ideally you don't want client to be aware of kubernetes also, it would be difficult to scope the access.
Re: Intelligent Kubernetes Load Balancing at Databricks
#13Less featureful than this, but we’ve been doing GRPC client side load balancing with kuberesolver[1] since 2018. It allows GRPC to handle the balancer implementations. It’s been rock solid for more than half a decade now. 1: https://github.com/sercand/kuberesolver
We use a headless service and client side load balancing for this. What's the difference ?
Re: Intelligent Kubernetes Load Balancing at Databricks
#14I wonder why they didn't use rendezvous hashing (aka HRW)[0]? It feels like it would solve all the requirement that they laid out, is fully client side, and doesn't require real time updates for the host list via discovery. [0] https://en.wikipedia.org/wiki/Rendezvous_hashing
HRW would cover the simple case, but they needed way more-- e.g. per-request balancing, zone affinity, live health checks, spillover, ramp-ups, etc. Once you need all that dynamic behavior, plain hashing just doesn’t cut it IMO. A custom client-side + discovery setup makes more sense.
Re: Intelligent Kubernetes Load Balancing at Databricks
#15Less featureful than this, but we’ve been doing GRPC client side load balancing with kuberesolver[1] since 2018. It allows GRPC to handle the balancer implementations. It’s been rock solid for more than half a decade now. 1: https://github.com/sercand/kuberesolver
What is the difference between Kuberesolver and using a Headless Service? In the README.md file, they compare it with a ClusterIP service, but not with a Headless on "ClusterIP: None". The advantages of using Kuberesolver are that you do not need to change DNS refresh and cache settings. However, I think this is preferable to the application calling the Kubernetes API.
I can give an n=1 anecdote here: the dns resolver used to have hard-coded caching which meant that it would be unresponsive to pod updates, and cause mini 30s outages.
The code in question was: https://github.com/grpc/grpc-go/blob/b597a8e1d0ce3f63ef8a7b6...
That meant that deploying a service which drained in less than 30s would have a little mini-outage for that service until the in-process DNS cache expired, with of course no way to configure it.
Kuberesolver streams updates, and thus lets clients talk to new pods almost immediately.
I think things are a little better now, but based on my reading of https://github.com/grpc/grpc/issues/12295, it looks like the dns resolver still might not resolve new pod names quickly in some cases.
Re: Intelligent Kubernetes Load Balancing at Databricks
#16Less featureful than this, but we’ve been doing GRPC client side load balancing with kuberesolver[1] since 2018. It allows GRPC to handle the balancer implementations. It’s been rock solid for more than half a decade now. 1: https://github.com/sercand/kuberesolver
We use a headless service and client side load balancing for this. What's the difference ?
instead of polling for endpoint updates, they're pushed to the client through k8s watches
Re: Intelligent Kubernetes Load Balancing at Databricks
#17Less featureful than this, but we’ve been doing GRPC client side load balancing with kuberesolver[1] since 2018. It allows GRPC to handle the balancer implementations. It’s been rock solid for more than half a decade now. 1: https://github.com/sercand/kuberesolver
kuberesolver is an interesting take as well. Directly watching the K8s API from each client could raise scaling concerns at very large scale, but it does open the door to using richer Kubernetes metadata for smarter load-balancing decisions. thanks for sharing!
if you don't want to expose k8s then there's the generic xds protocol
Re: Intelligent Kubernetes Load Balancing at Databricks
#18Less featureful than this, but we’ve been doing GRPC client side load balancing with kuberesolver[1] since 2018. It allows GRPC to handle the balancer implementations. It’s been rock solid for more than half a decade now. 1: https://github.com/sercand/kuberesolver
[deleted]
Re: Intelligent Kubernetes Load Balancing at Databricks
#19Less featureful than this, but we’ve been doing GRPC client side load balancing with kuberesolver[1] since 2018. It allows GRPC to handle the balancer implementations. It’s been rock solid for more than half a decade now. 1: https://github.com/sercand/kuberesolver
kuberesolver is an interesting take as well. Directly watching the K8s API from each client could raise scaling concerns at very large scale, but it does open the door to using richer Kubernetes metadata for smarter load-balancing decisions. thanks for sharing!
[deleted]
Re: Intelligent Kubernetes Load Balancing at Databricks
#20Less featureful than this, but we’ve been doing GRPC client side load balancing with kuberesolver[1] since 2018. It allows GRPC to handle the balancer implementations. It’s been rock solid for more than half a decade now. 1: https://github.com/sercand/kuberesolver
I've been using a standardized xds resolver[1]. The benefit here is that you don't have to patch grpc clients.