Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLB: GitHub's open source load balancer

githubengineering.com

Post reply on HN
Print view View original

21–30 of 65 posts

Re: GLB: GitHub's open source load balancer

#21

Post by lmb »

justinsaccount wrote: post #16
lmb wrote: post #11

Earlier quoted context omitted.

I haven't looked at what Google has released, but there are big differences between GLB and Katran. (Not affiliated with any of those companies) In terms of technology, Katran uses XDP and IPIP tunnels, both upstream in the Linux kernel. GLB uses DPDK, which allows processing raw packets in user space, and Geberic UDP encapsulation + a custom iptables module. Neither DPDK nor the module are upstream. There are archit…

> uses connection tracking at the load balancer to know where to send packets for established flows Kind of, from katran post: > Each L4LB also stores the backend choice for each 5-tuple as a lookup table to avoid duplicate computation of the hash on future packets. This state is a pure optimization and is not necessary for correctness > .. > Katran uses an extended version of the Maglev hash to select the backend se…

If the connection tracking state is really not required I don't understand how Katran works. How does it deal with the set of live backends changing? Using Maglev hashing gives you "minimal disruption", not no disruption.

Re: GLB: GitHub's open source load balancer

#22

Post by drewpc »

jsiepkes wrote: post #3

Looks really cool! Though a simpler solution for most people will probably be OpenBSD's CARP protocol to share a single virtual IP between multiple boxes (with for example relayd). ECMP routing can get complex fast.

Also note that CARP, VRRP, HSRP, and GLBP are all layer 3 load balancing protocols (that rely on layer 2 magic to work) whereas this is a layer 4 load balancer. Meaning I can connect to a single IP and a single port (80) and get consistently and equitably load balanced to exactly one server (of many) at another IP address.

Re: GLB: GitHub's open source load balancer

#24

Post by llama052 »

Maybe I don't see the use case since I'm not at that scale, but it seems like a lot of added complexity for what appears to be hacking around using other load balancing solutions as a Layer-4 option?

Edit: It's a question, if you downvote please let me know why it's a better solution.

Re: GLB: GitHub's open source load balancer

#25

Post by KenanSulayman »

atombender wrote: post #14
KenanSulayman wrote: post #12

Why would one use this over HAProxy?

HAProxy is an Layer 7 (i.e. HTTP, for the most part) load balancer and only handles the use case of spreading load across multiple backends. A single instance binds to a single IP. There's no redundancy; lose the HAProxy and you lose traffic. For true redundancy, you need a layer above that handles the distribution of traffic to multiple redundant load balancer instances, and GLB does that via ECMP (Equal-Cost Multi-…

I have very successfully in the past and still do use HAProxy as level 4 LB. It's one of the fastest to my knowledge. I have used HAProxy as entry to big Mesos clusters without any issue before.

One example of using HAProxy as L4 LB instead of letting it do the termination is when it is proxying TLS traffic from and to multiple backends. Or Websocket. Or even as bastion LB for SSH should one bastion go down.

Re: GLB: GitHub's open source load balancer

#26

Post by KenanSulayman »

amanzi wrote: post #19
KenanSulayman wrote: post #12

Why would one use this over HAProxy?

From the article: >GLB Director does not replace services like haproxy and nginx, but rather is a layer in front of these services (or any TCP service) that allows them to scale across multiple physical machines without requiring each machine to have unique IP addresses.

Yes, I get that. But in Mesos, for instance, you could use overlay-networks to get one virtual IP per HAProxy cluster (i.e. 3 HAProxy instances). They are then round-robin'd on each server in the whole cluster so that you have a transparent LB with a singular IP.

Re: GLB: GitHub's open source load balancer

#27

Post by atombender »

KenanSulayman wrote: post #25
atombender wrote: post #14

Earlier quoted context omitted.

HAProxy is an Layer 7 (i.e. HTTP, for the most part) load balancer and only handles the use case of spreading load across multiple backends. A single instance binds to a single IP. There's no redundancy; lose the HAProxy and you lose traffic. For true redundancy, you need a layer above that handles the distribution of traffic to multiple redundant load balancer instances, and GLB does that via ECMP (Equal-Cost Multi-…

I have very successfully in the past and still do use HAProxy as level 4 LB. It's one of the fastest to my knowledge. I have used HAProxy as entry to big Mesos clusters without any issue before. One example of using HAProxy as L4 LB instead of letting it do the termination is when it is proxying TLS traffic from and to multiple backends. Or Websocket. Or even as bastion LB for SSH should one bastion go down.

It's not that HAProxy doesn't do L4. As I said, projects like GLB solves how to make the load balancer itself redundant; how to load balance the load balancer, so to speak.

Re: GLB: GitHub's open source load balancer

#28

Post by atombender »

KenanSulayman wrote: post #26
amanzi wrote: post #19

Earlier quoted context omitted.

From the article: >GLB Director does not replace services like haproxy and nginx, but rather is a layer in front of these services (or any TCP service) that allows them to scale across multiple physical machines without requiring each machine to have unique IP addresses.

Yes, I get that. But in Mesos, for instance, you could use overlay-networks to get one virtual IP per HAProxy cluster (i.e. 3 HAProxy instances). They are then round-robin'd on each server in the whole cluster so that you have a transparent LB with a singular IP.

Round-robin routing is one way that really doesn't cut it at the level of scale that Github and others operate at. Overlay networks also typically have no notion of load and how to send traffic to the hop with the least amount of load.

Re: GLB: GitHub's open source load balancer

#29

Post by notyourday »

lmb wrote: post #11
justinsaccount wrote: post #5

See also: Facebook: https://code.fb.com/open-source/open-sourcing-katran-a-scala... Google: https://cloudplatform.googleblog.com/2016/03/Google-shares-s... The design of all 3 is very similar.

I haven't looked at what Google has released, but there are big differences between GLB and Katran. (Not affiliated with any of those companies) In terms of technology, Katran uses XDP and IPIP tunnels, both upstream in the Linux kernel. GLB uses DPDK, which allows processing raw packets in user space, and Geberic UDP encapsulation + a custom iptables module. Neither DPDK nor the module are upstream. There are archit…

It is unfortunate that Fastly did not open source faild. That's the most elegant solution to draining that I have ever seen.

Re: GLB: GitHub's open source load balancer

#30

Post by llama052 »

atombender wrote: post #27
KenanSulayman wrote: post #25

Earlier quoted context omitted.

I have very successfully in the past and still do use HAProxy as level 4 LB. It's one of the fastest to my knowledge. I have used HAProxy as entry to big Mesos clusters without any issue before. One example of using HAProxy as L4 LB instead of letting it do the termination is when it is proxying TLS traffic from and to multiple backends. Or Websocket. Or even as bastion LB for SSH should one bastion go down.

It's not that HAProxy doesn't do L4. As I said, projects like GLB solves how to make the load balancer itself redundant; how to load balance the load balancer, so to speak.

For the cost of a ton of added complexity though? What do you get out of this solution that other solutions don't provide? Say DNS load-balancing, VRRP, CARP, or any other HA solution.
Post reply on HN