Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The GitHub Load Balancer

githubengineering.com

Post reply on HN
Print view View original

91–100 of 137 posts

Re: The GitHub Load Balancer

#91

Post by sytse »

dogismycopilot wrote: post #90
sytse wrote: post #83

Earlier quoted context omitted.

:) We're not doing this better. We're struggling with our load balancers right now. We're using Azure load balancers and then HAproxy. But the Azure ones sometimes don't work. Luckily the new network type on Azure supports floating IPs so we can set something up ourselves https://gitlab.com/gitlab-com/infrastructure/issues/466

I would love to see the solution. We're also desiring to run HAProxy in Azure with keepalived (even in unicast mode). The black-box "windows based" load balancer that Azure offers is quite limited.

Cool, we're trying to do all the infrastructure work in the open under https://gitlab.com/groups/gitlab-com

Re: The GitHub Load Balancer

#92

Post by jssjr »

bogomipz wrote: post #76
jssjr wrote: post #40

Earlier quoted context omitted.

Anycast usually implies traffic will be directed to the nearest node advertising that prefix. The GLB directors leverage ECMP which provides the ability to balance flows across many available paths.

Anycast and ECMP work together in the context of load balancing. ECMP without Anycasted destination IPs would be pointless for horizontally scaling your LB tier. What Anycast means is just that multiple hosts share the same IP address - as opposed to unicast. When all the nodes sharing the same IP are on the same subnet "nearest" is kind of irrelevant. So the implication is different.

Sure. Feel free to call it anycast then. I usually hear anycast routing used in the context of achieving failover or routing flows to the closest server/POP, but there is probably a more formal definition in an RFC that I'll be pointed to shortly. =)

We are using BGP to advertise prefixes for GLB inside the data center to route flows to the directors. In our case all of the nodes are not on the same subnet (or at least not guaranteed to be) which is one of the reasons why we chose to avoid solutions requiring multicast. I expect Joe and Theo will get into more details about that in a future post though.

Re: The GitHub Load Balancer

#93

Post by gumby »

wmf wrote: post #74
gumby wrote: post #69

They talk about running on "bare metal" but when I followed that link it looked like they were simply running under Ubuntu. Is it so much a given that everything is going to be virtualized? When I think of "bare metal" I think of a single image with disk management, network stack, and what few services they want all running in supervisory mode. Basically the architecture of an embedded system.

Yes, it is assumed that all startups are running in EC2 us-east-1 and "bare metal" is the accepted term for non-virtualized systems.

Thanks for the clarification. Yet another dilution of a technical term, sigh.

I was quite excited when I read it, and felt quite let down when I followed up.

Re: The GitHub Load Balancer

#94

Post by gwright »

While I understand that NIH syndrome is a real thing, it is very dissapointing to read many of the comments here.

I think very few HN readers are really in a position to have an informed opinion regarding Github's decision to build new piece of software rather than using an existing system.

Personally I find this area quite interesting to read about because it is very difficult to build highly available, scalable, and resilient network service endpoints. Plain old TCP/IP isn't really up to the job. Dealing with this without any cooperation from the client side of the connection adds to the difficulty.

I look forward to hearing more about GLB.

Re: The GitHub Load Balancer

#95

Post by paulddraper »

polpo wrote: post #77
ckdarby wrote: post #68

Earlier quoted context omitted.

Have you ever contributed to HaProxy? Have you ever tried committing massive alterations to major open source projects? It isn't as simple as here's my massive rewrite, click the accept button and everything works out for the open source community. Let me be the first to say that the level of politics, circle jerking and knowing people is ridiculous.

Given the good reaction to an out-of-the-blue patch from me on the HAProxy mailing list, I'd imagine that contributing even major changes to HAProxy probably would go rather well. It's one of the best open source development communities I've experienced. Welcoming, but still highly focused on quality contributions. The quality and performance of HAProxy reflects this approach.

HAProxy is exceptionally good in this regard.

Re: The GitHub Load Balancer

#96

Post by paulddraper »

gjvc wrote: post #50
jimjag wrote: post #29

I am increasingly bothered by the "not invented here" syndrome where instead of taking existing projects and enhancing them, in true open source fashion, people instead re-create from scratch. It is then justified that their creation is needed because "no one else has these kinds of problems" but then they open source them as if lots of other people could benefit from it. Why open source something if it has an expect…

by that rationale, Apache httpd would have stayed Apache httpd, and we wouldn't have gotten nginx.

Shudder.

Re: The GitHub Load Balancer

#97

Post by sitkack »

positr0n wrote: post #81
olalonde wrote: post #57

Earlier quoted context omitted.

> Due to their unique requirements [...] This is both plausible I'm skeptical that Github's load balancing requirements are that different from any of the other large file hosts and SaaS companies. But it's possible and I'm not in a position to tell. That being said, the "NIH syndrome" is a largely overlooked problem in our industry and I think it's reasonable to raise concerns over new projects that may be reinventi…

Some of the requirements they list are pretty unique to github since they serve .git to git clients as well as http to http clients (which is what most SaaS companies do). Like a very long running git clone from someone with slow internet not having its connection dropped.

As person who has used github on totally lousy connections in remote parts of the world, it faired far better than most Isomorphic web apps. People in Nepal can use github to actually get work done. Most current generation web apps won't even load.

Re: The GitHub Load Balancer

#98

Post by tadelle »

dpcx wrote: post #84
falcolas wrote: post #38

Earlier quoted context omitted.

My two cents? Because they have based their entire business and clientele around an open source piece of technology. Their contributions back to the community which gave them the core of their company feels... anemic. Even if their entire stack was open sourced, GitHub would still be a profitable company simply because it's GitHub. They're hosting hundreds of thousands of repos on bare metal and providing tools aroun…

By that definition Google and Facebook should do the same. Or asking a farmer to give away their seeds for a similar reason.

As I can see you and most of the people in this thread are don't get the basic idea and philosophy of open-source software. GitHub supports open-source with closed-source software. Their main job is based on supporting open-source software not being a search engine or social network. That's why this move is not mandatory for Google and Facebook. If you want to support something you should take the first step otherwise I could say you are dishonest to yourself, your ideology and to people who wants to support same philosophy from their hearts. There is no difference between the GitHub I mentioned and politician that uses religion to become much more powerful using folks belief.

Re: The GitHub Load Balancer

#99

Post by Retra »

serge2k wrote: post #86
jimjag wrote: post #29

I am increasingly bothered by the "not invented here" syndrome where instead of taking existing projects and enhancing them, in true open source fashion, people instead re-create from scratch. It is then justified that their creation is needed because "no one else has these kinds of problems" but then they open source them as if lots of other people could benefit from it. Why open source something if it has an expect…

> I am increasingly bothered by the "not invented here" syndrome I'm bothered by the increasing prevalence of "never invent here". FOSS is great, but if it's not meeting your needs then writing your own is perfectly valid.

And often one of your needs is "have people on hand who know exactly how it works", and the easiest way to achieve that is to have those people build it from scratch. That's the only way to actually ensure it makes your needs priority #1.

Re: The GitHub Load Balancer

#100

Post by bogomipz »

jssjr wrote: post #92
bogomipz wrote: post #76

Earlier quoted context omitted.

Anycast and ECMP work together in the context of load balancing. ECMP without Anycasted destination IPs would be pointless for horizontally scaling your LB tier. What Anycast means is just that multiple hosts share the same IP address - as opposed to unicast. When all the nodes sharing the same IP are on the same subnet "nearest" is kind of irrelevant. So the implication is different.

Sure. Feel free to call it anycast then. I usually hear anycast routing used in the context of achieving failover or routing flows to the closest server/POP, but there is probably a more formal definition in an RFC that I'll be pointed to shortly. =) We are using BGP to advertise prefixes for GLB inside the data center to route flows to the directors. In our case all of the nodes are not on the same subnet (or at lea…

Are you running Quagga or Bird on the director instances then? I'm looking forward to reading more about it.
Post reply on HN