Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub hit by DDoS attack second day in a row

thenextweb.com

Post reply on HN
Print view View original

61–70 of 72 posts

Re: GitHub hit by DDoS attack second day in a row

#61

Post by bkanber »

Tobu wrote: post #48
bkanber wrote: post #33

Earlier quoted context omitted.

Devil's avocado here: let's say you pay $100/mo for a gym membership and they shut down three days in a row because somebody called in a threat. How upset would you be at the gym? A malicious attack by a third party is different from, say, the gym allowing black mold to grow in the locker room. I'd quit a gym if they had black mold. That's mismanagement. I wouldn't quit a gym if malicious third party intervention inc…

What makes DDOSes different from black mold? Both are expected risks and should be mitigated. Yeah, there are sentient actors behind the DDOS, but GitHub has to deal with it at the level of their infrastructure either way.

The fact that there are sentient actors behind the DDoS _is_ the difference.

You can reliably predict and protect against things like network outages, server failures, full datacenter failures (black mold)--you can directly measure their impact and plan failover paths. A DB server goes out? Whatever! That's why you have a hot backup or two online and ready to go.

What you can't predict is exactly how far a malicious third party will go to hurt you. You can't predict how many dollars they'll spend on their botnet minutes. You don't know if they're going to attack your infrastructure or the DNS. Can buying more bandwidth fix the problem? If so, how much more? And will the attacker simply up the ante when they see that you're recovering? Can filtering requests fix the problem? If so, will the attacker provision different resources to attack you with?

This isn't simply a matter of infrastructure, buying the right equipment, or setting things up "just right" precisely because there is a sentient actor trying to hurt you. It's more like a game of chess.

Re: GitHub hit by DDoS attack second day in a row

#62

Post by nnnnnnnn »

Firehed wrote: post #31
codinghorror wrote: post #23

Can we please use some of that 100 million investment to buy an infrastructure that is more resilient to these kinds of DDoS attacks? Pretty please? This is a service I pay for and my business relies on. Having it down three times in three days impacts our work.

Easier said than done. We've had DDOS issues in the past as well, and getting it resolved - even by throwing money at the problem - is nontrivial. What amounts to throwing a massive amount of hardware at the problem (i.e., boxes that can handle 10-100+gbps of traffic, filter out the attacks, and pass only legit stuff down to your servers) is expensive[1], and casuses all sorts of unexpected behavior: API clients myst…

Github can easily afford to use someone like Prolexic. And they should.

When you say things like "And it's all dependent on attackers not getting the IP of your actual servers" this makes me wonder how much you understand the subject matter. There are many, many options.

Re: GitHub hit by DDoS attack second day in a row

#63

Post by alaaibrahim »

didip wrote: post #54
richadams wrote: post #25

Earlier quoted context omitted.

SVN's too good for 'em.. let them use Visual Source Safe forever.

Visual Source Safe is too good for 'em.. may they be doomed to emailing .patch files forever.

Emailing .patch files is better than VSS :-)

Re: GitHub hit by DDoS attack second day in a row

#64

Post by feisuzhu »

emeraldd wrote: post #24
carlosaguayo wrote: post #17

Earlier quoted context omitted.

may whoever is doing this be doomed to use SVN the rest of their lives ...

Forget svn ... May they be doomed to use RCS for the rest of their days ...

Forget about version control, may them be using pen drives to copy code around for the rest of their days...

Re: GitHub hit by DDoS attack second day in a row

#65

Post by Stratoscope »

carlosaguayo wrote: post #17
redsymbol wrote: post #6

Maybe it's because I just personally identify with the founders of github (i.e. entrepreneurial sw engineers), but I'm starting to get mad at whoever keeps doing this. Here's hoping that with all the smart people this is affecting, the people responsible will be tracked down and exposed.

may whoever is doing this be doomed to use SVN the rest of their lives ...

May they be doomed to get a call like I did some 20 years ago from a large software company I'd consulted for:

"Mike, remember that project you did for us last year? Yeah, we've been shipping it with our product for a while - working great, thanks! Say... I don't suppose you might happen to still have a copy of the source code anywhere? I know, you probably deleted it after the project was over, but you were always so good about making backups - do you suppose you could rummage through your old backup tapes and see if there's anything? No, it's nothing like that! Well, we seem to have lost all our copies of the source and really hope you could help us out."

True story.

Re: GitHub hit by DDoS attack second day in a row

#66

Post by bitcartel »

zalambar wrote: post #44
peripetylabs wrote: post #38

Perhaps I'm misunderstanding: I thought one goal of DVCS was to remove central points of failure? In that sense, isn't a central "hub" regressive? I wonder if there's a way to host Git repositories with static files, say, on Amazon S3... That would be neat.

Git is distributed and there's no reason you should have to stop working, or committing, just because github is temporarily unavailable. At least for dependencies only on git. Losing access to wikis, pull requests, and issues may be a problem for some teams. http://ozmm.org/posts/when_github_goes_down.html has a good summary of quick ways to keep using git without github.

Fossil has built-in issues and wiki. It just never became popular.

Re: GitHub hit by DDoS attack second day in a row

#67

Post by Firehed »

nnnnnnnn wrote: post #62
Firehed wrote: post #31

Earlier quoted context omitted.

Easier said than done. We've had DDOS issues in the past as well, and getting it resolved - even by throwing money at the problem - is nontrivial. What amounts to throwing a massive amount of hardware at the problem (i.e., boxes that can handle 10-100+gbps of traffic, filter out the attacks, and pass only legit stuff down to your servers) is expensive[1], and casuses all sorts of unexpected behavior: API clients myst…

Github can easily afford to use someone like Prolexic. And they should. When you say things like "And it's all dependent on attackers not getting the IP of your actual servers" this makes me wonder how much you understand the subject matter. There are many, many options.

Prolexic's servers don't take the load if the attackers know where the computers behind the scrubbers are. Configuring iptables to ignore all traffic not coming from prolexic's IPs doesn't come close to fending off a DDOS.

I know this because I was told this by prolexic while configuring our servers to sit behind their scrubbing servers while we're under an equally crippling DDOS (one that took down half the customers in our datacenter, not just us). So while I haven't examined their tech stack under a magnifying glass, I'm not exactly talking out of my ass here.

Yes, there are other options but those don't take an hour to implement like signing a contract and changing a few DNS entries does. And when these conditions exist, you need an answer that can be implemented in an hour.

Re: GitHub hit by DDoS attack second day in a row

#68

Post by nnnnnnnn »

Firehed wrote: post #67
nnnnnnnn wrote: post #62

Earlier quoted context omitted.

Github can easily afford to use someone like Prolexic. And they should. When you say things like "And it's all dependent on attackers not getting the IP of your actual servers" this makes me wonder how much you understand the subject matter. There are many, many options.

Prolexic's servers don't take the load if the attackers know where the computers behind the scrubbers are. Configuring iptables to ignore all traffic not coming from prolexic's IPs doesn't come close to fending off a DDOS. I know this because I was told this by prolexic while configuring our servers to sit behind their scrubbing servers while we're under an equally crippling DDOS (one that took down half the customer…

You are fabricating straw men. They do not need "an answer that can be implemented in an hour." They have been in business for 4 years, and this particular string of DDoS attacks has been going on for several days now. This is both a a planning failure and an incident response failure.

Your comment about iptables is odd. I don't know why iptables would be relevant here; I suspect we are talking about implementations several orders of magnitude different in size. Certainly one would drop traffic at the edges and not do filtering on end nodes.

Re: GitHub hit by DDoS attack second day in a row

#69

Post by threedaymonk »

bitcartel wrote: post #66
zalambar wrote: post #44

Earlier quoted context omitted.

Git is distributed and there's no reason you should have to stop working, or committing, just because github is temporarily unavailable. At least for dependencies only on git. Losing access to wikis, pull requests, and issues may be a problem for some teams. http://ozmm.org/posts/when_github_goes_down.html has a good summary of quick ways to keep using git without github.

Fossil has built-in issues and wiki. It just never became popular.

The only prominent Fossil user I'm aware of is Zed Shaw, and he's since recanted after losing data and having trouble upgrading: http://sheddingbikes.com/posts/1306005291.html

Re: GitHub hit by DDoS attack second day in a row

#70

Post by peripetylabs »

mattdeboard wrote: post #57
peripetylabs wrote: post #38

Perhaps I'm misunderstanding: I thought one goal of DVCS was to remove central points of failure? In that sense, isn't a central "hub" regressive? I wonder if there's a way to host Git repositories with static files, say, on Amazon S3... That would be neat.

>Perhaps I'm misunderstanding: I thought one goal of DVCS was to remove central points of failure? In that sense, isn't a central "hub" regressive? This meme is getting really, really tiresome. Github being down is NOT a central point of failure. Most people know that setting up your own git server is trivial, literally a 3-4 step process. We know that we don't lose our files, our history, our working tree, etc. The…

So it seems we agree. I was asking: why use Github at all? Most of the value added of Github can be done with CSS and simple network analysis.
Post reply on HN