Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLB: GitHub's open source load balancer

githubengineering.com

Post reply on HN
Print view View original

11–20 of 65 posts

Re: GLB: GitHub's open source load balancer

#11

Post by lmb »

justinsaccount wrote: post #5

See also: Facebook: https://code.fb.com/open-source/open-sourcing-katran-a-scala... Google: https://cloudplatform.googleblog.com/2016/03/Google-shares-s... The design of all 3 is very similar.

I haven't looked at what Google has released, but there are big differences between GLB and Katran. (Not affiliated with any of those companies)

In terms of technology, Katran uses XDP and IPIP tunnels, both upstream in the Linux kernel. GLB uses DPDK, which allows processing raw packets in user space, and Geberic UDP encapsulation + a custom iptables module. Neither DPDK nor the module are upstream.

There are architectural differences as well. Katran is much closer to a classic load balancer, and uses connection tracking at the load balancer to know where to send packets for established flows. GLB has no per-flow state at the load balancer, which gives it the very nice property that load balancers can be added and removed from an ECMP group without disturbing existing connections. There is an academic paper about a system called Beamer, which most likely influenced GLB (or maybe the other way round?). It's a good read, and relatively short.

Finally, Katran is really a C++ library you could build a load balancer on, while GLB comes with batteries included.

I think GLB looks nice, hats off to GitHub for open sourcing it.

Re: GLB: GitHub's open source load balancer

#14

Post by atombender »

KenanSulayman wrote: post #12

Why would one use this over HAProxy?

HAProxy is an Layer 7 (i.e. HTTP, for the most part) load balancer and only handles the use case of spreading load across multiple backends. A single instance binds to a single IP. There's no redundancy; lose the HAProxy and you lose traffic.

For true redundancy, you need a layer above that handles the distribution of traffic to multiple redundant load balancer instances, and GLB does that via ECMP (Equal-Cost Multi-Path) routing. Github supposedly uses HAProxy as their L7 load balancer.

All of this thoroughly explained in the article.

Re: GLB: GitHub's open source load balancer

#15

Post by parthdesai »

KenanSulayman wrote: post #12

Why would one use this over HAProxy?

>GLB Director does not replace services like haproxy and nginx, but rather is a layer in front of these services (or any TCP service) that allows them to scale across multiple physical machines without requiring each machine to have unique IP addresses.

It's right there in 2nd paragraph.

Re: GLB: GitHub's open source load balancer

#16

Post by justinsaccount »

lmb wrote: post #11
justinsaccount wrote: post #5

See also: Facebook: https://code.fb.com/open-source/open-sourcing-katran-a-scala... Google: https://cloudplatform.googleblog.com/2016/03/Google-shares-s... The design of all 3 is very similar.

I haven't looked at what Google has released, but there are big differences between GLB and Katran. (Not affiliated with any of those companies) In terms of technology, Katran uses XDP and IPIP tunnels, both upstream in the Linux kernel. GLB uses DPDK, which allows processing raw packets in user space, and Geberic UDP encapsulation + a custom iptables module. Neither DPDK nor the module are upstream. There are archit…

> uses connection tracking at the load balancer to know where to send packets for established flows

Kind of, from katran post:

> Each L4LB also stores the backend choice for each 5-tuple as a lookup table to avoid duplicate computation of the hash on future packets. This state is a pure optimization and is not necessary for correctness

> ..

> Katran uses an extended version of the Maglev hash to select the backend server

Re: GLB: GitHub's open source load balancer

#19

Post by amanzi »

KenanSulayman wrote: post #12

Why would one use this over HAProxy?

From the article:

>GLB Director does not replace services like haproxy and nginx, but rather is a layer in front of these services (or any TCP service) that allows them to scale across multiple physical machines without requiring each machine to have unique IP addresses.

Re: GLB: GitHub's open source load balancer

#20

Post by llama052 »

atombender wrote: post #14
KenanSulayman wrote: post #12

Why would one use this over HAProxy?

HAProxy is an Layer 7 (i.e. HTTP, for the most part) load balancer and only handles the use case of spreading load across multiple backends. A single instance binds to a single IP. There's no redundancy; lose the HAProxy and you lose traffic. For true redundancy, you need a layer above that handles the distribution of traffic to multiple redundant load balancer instances, and GLB does that via ECMP (Equal-Cost Multi-…

I thought best practice for HAproxy was to run two HAProxy's in parallel with VRRP or DNS load balancing?

Does that not achieve the same outcome? I've used HAProxy for layer 4 in the past without any issue this way.

Post reply on HN