Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The GitHub Load Balancer

githubengineering.com

Post reply on HN
Print view View original

61–70 of 137 posts

Re: The GitHub Load Balancer

#61

Post by the_duke »

fps wrote: post #51
the_duke wrote: post #39

Earlier quoted context omitted.

Agreed. Considering the amazing value Github provides for the IT world, I really don't care if they release their code. Isn't it enought that they have revolutionized development workflows, project hosting and made contributing and collaborating so much easiert? (to be clear, GH wasn't the first site of it's kind, but the network effect of having one site that that basically every developer uses daily -and be it just…

I use github every day for professional and personal projects, and typically like what they've done, but Sourceforge.net was the github of the early 2000's, where almost every developer had an account and where you could get free development and distribution tools for your project. Look at what it is now (or was before it was re-opened a few months ago.) People's open source projects were being leveraged to push malw…

Mhm, I used Sourceforge from time to time back then.

But I remember it as always clumsy (even before they got bought and started the ad craziness), and only a small subset of projects was on Sourceforge, in my experience at least.

Re: The GitHub Load Balancer

#63

Post by logicalstack »

NatW wrote: post #60

I'm curious if they looked into pf / CARP as part of their research into allowing horizontal scalability for an ip. See: https://www.openbsd.org/faq/pf/carp.html

CARP and similar systems require an active/passive configuration which we did not want since it needs at least twice as many hosts, half of which are not doing any work. We had similar issues with our former Git storage system based on DRDB (http://githubengineering.com/introducing-dgit/).

pfsync, lvs and etc uses multicast to share connection state which we also wanted to avoid.

Re: The GitHub Load Balancer

#64

Post by dismantlethesun »

jimjag wrote: post #29

I am increasingly bothered by the "not invented here" syndrome where instead of taking existing projects and enhancing them, in true open source fashion, people instead re-create from scratch. It is then justified that their creation is needed because "no one else has these kinds of problems" but then they open source them as if lots of other people could benefit from it. Why open source something if it has an expect…

For me, NIH is about the idea of already having deep knowledge of the problem domain so a solution is relatively straight forwards. Sure it'll take X time, and there might be some hiccups, but its all about effort and not having to learn anything new.

Joining a pre-existing project is more reasonable when you just can't replicate the basis of the project without learning a lot of new things.

That's the reason why there's a plethora of compile to JS languages, but only a few actual javascript virtual machines.

Re: The GitHub Load Balancer

#65

Post by dismantlethesun »

gm-conspiracy wrote: post #44
sanderjd wrote: post #27

Earlier quoted context omitted.

I have never worked on a piece of proprietary software that would be in a state where it could be released publicly in any form without a very large outlay of development time and money. I'd be very surprised if github is any different. "Why not" is for things that are easy to do. Things that are hard and / or expensive to do require strong justification.

Honest question - do you think if the proprietary software you worked on had been planned from the beginning to be released publicly, a very large outlay would be unnecessary?

Not him, but if I were writing something that was going to be publicly released then I could imagine the team would:

a) Have to include a lot more documentation

b) Be a lot more paranoid about best practices, and writing generalizable code even if it doesn't fit their particular slice of the market

c) Never have the chance to use a closed source solution for part of the problem (e.g. no we can't have half our app actually be a series of Oracle specific stored procedures, or buy a UI package for $20,000 rather than use a heavily modified version of Bootstrap).

Re: The GitHub Load Balancer

#66

Post by audleman »

lifeisstillgood wrote: post #14

I love using GitHub and appreciate the impact it is and has had. But this post is what is wrong with the web today. They have taken a distributed-at-it's-plumbing technology, and centralised it so much that now we need to innovate new load balancing mechanisms. Years ago I worked at Demon Internet and we tried to give every dial up user a piece of webspace - just a disk always connected. Almost no one ever used them.…

Are you saying instead of having Github, we should all be hosting our own Git repos?

> But it is beginning to creak. GitHub faces scaling challenges,

I don't agree that Github facing scaling issues means the web is creaking. More like old wooden boats are being replaced by big, sturdy battleships. I think the web is getting stronger thanks to engineers facing the challenges coming their way.

> I am frustrated that some people are on whatsapp and some slack and some telegram, and I cannot track who is talking to me.

If you're annoyed by people messaging on you through multiple platforms, it seems the solution would be to only have one provider. But you earlier call that "what is wrong with the web today," and that we should have distributed systems.

Re: The GitHub Load Balancer

#67

Post by MichaelGG »

tomschlick wrote: post #21
tadelle wrote: post #15

Earlier quoted context omitted.

But they could release trimmed community version at least ;)

Why? Open sourcing GH would probably be a huge undertaking for a considerable amount of their team (removing proprietary code, making sure no credentials are in there, etc).

They already have a version they ship to customers (Enterprise) so presumably they have done the work of removing secrets.

Re: The GitHub Load Balancer

#68

Post by ckdarby »

jimjag wrote: post #29

I am increasingly bothered by the "not invented here" syndrome where instead of taking existing projects and enhancing them, in true open source fashion, people instead re-create from scratch. It is then justified that their creation is needed because "no one else has these kinds of problems" but then they open source them as if lots of other people could benefit from it. Why open source something if it has an expect…

Have you ever contributed to HaProxy? Have you ever tried committing massive alterations to major open source projects?

It isn't as simple as here's my massive rewrite, click the accept button and everything works out for the open source community.

Let me be the first to say that the level of politics, circle jerking and knowing people is ridiculous.

Re: The GitHub Load Balancer

#69

Post by gumby »

They talk about running on "bare metal" but when I followed that link it looked like they were simply running under Ubuntu. Is it so much a given that everything is going to be virtualized?

When I think of "bare metal" I think of a single image with disk management, network stack, and what few services they want all running in supervisory mode. Basically the architecture of an embedded system.

Re: The GitHub Load Balancer

#70

Post by senko »

ckdarby wrote: post #68
jimjag wrote: post #29

I am increasingly bothered by the "not invented here" syndrome where instead of taking existing projects and enhancing them, in true open source fashion, people instead re-create from scratch. It is then justified that their creation is needed because "no one else has these kinds of problems" but then they open source them as if lots of other people could benefit from it. Why open source something if it has an expect…

Have you ever contributed to HaProxy? Have you ever tried committing massive alterations to major open source projects? It isn't as simple as here's my massive rewrite, click the accept button and everything works out for the open source community. Let me be the first to say that the level of politics, circle jerking and knowing people is ridiculous.

"Contributing to open source is hard, let's write our own and open source it instead."
Post reply on HN