Live data from Hacker News

Posterous is being DDoS'd

twitter.com

21–30 of 49 posts

Re: Posterous is being DDoS'd

#21
post #20

We're getting about 500k packets per second, 500mbps to 1.5gbps peak, it's a synflood from a botnet. Typically we can IP hop and null-route the old IP's. That usually buys us about a day until the botnet phones home to get the new IP's, at which point we just hop again. Since our DNS TTL is only 5 minutes at most we are down 5 minutes. TODAY, the attackers hopped IP's to our new IP immediately. So they appear to be l…

Any suggestions for customers with DNS names pointing to posterous blogs? Are we going to need to follow you on each hop?

Custom domains are unaffected, no action needed.

Re: Posterous is being DDoS'd

#22
post #20

We're getting about 500k packets per second, 500mbps to 1.5gbps peak, it's a synflood from a botnet. Typically we can IP hop and null-route the old IP's. That usually buys us about a day until the botnet phones home to get the new IP's, at which point we just hop again. Since our DNS TTL is only 5 minutes at most we are down 5 minutes. TODAY, the attackers hopped IP's to our new IP immediately. So they appear to be l…

Any suggestions for customers with DNS names pointing to posterous blogs? Are we going to need to follow you on each hop?

I wouldn't think so...

Re: Posterous is being DDoS'd

#24
post #7

This is hot off the heels of an email sent out last night: As you’re no doubt aware, Posterous has had a rocky six days. On Wednesday and Friday, our servers were hit by massive Denial of Service (DoS) attacks. We responded quickly and got back online within an hour, but it didn’t matter; the site went down and our users couldn’t post. On Friday night, our team worked around the clock to move to new data centers, bet…

This sort of crisis management is always a challenge. How do you communicate this sort of problem to customers without losing credibility later if there is a risk of lingering issues? Based on the tone of the email that you posted, as a customer, I would expect the problem solved, case closed. Certainly, it makes customers feel good that the team has it handled. The new events, if not handled quickly, could create do…

Simple rule about DDoSes: there are no guarantees.

Re: Posterous is being DDoS'd

#26

I seriously doubt this is the reason but they insulted a lot of people with their "switch to us" targeted attacks. That was classless.

It wasn't classless, it was marketing. No one wants their competitors poaching their clients but in business you deal with it. That said, the same thought went through my mind, I wonder if it's related.

Re: Posterous is being DDoS'd

#27

We're getting about 500k packets per second, 500mbps to 1.5gbps peak, it's a synflood from a botnet. Typically we can IP hop and null-route the old IP's. That usually buys us about a day until the botnet phones home to get the new IP's, at which point we just hop again. Since our DNS TTL is only 5 minutes at most we are down 5 minutes. TODAY, the attackers hopped IP's to our new IP immediately. So they appear to be l…

Do all the caching name servers respect the TTL though?

Re: Posterous is being DDoS'd

#28
post #17
post #5

Our datacenter is experiencing heavy packet loss. We're on the line with Rackspace now. I don't see where they stated it was a DDOS attack. Packet loss can occur due to a large number of different issues. EDIT: They just clarified with the following: The DDoS attackers have returned and evolved their attack around our countermeasures. We expect to be back online ASAP w/ @gigenet antiddos

It's interesting that they're basically saying Rackspace's network couldn't hold up and doesn't have an effective solution for DDOS victims on their network. Gige's DDOS protection is basically a hosted redirect and filtering system that "leverage[s] the cost of DDoS mitigation amongst a large group of businesses, giving you access to the infrastructure that would normally be out of reach financially." http://www.gig…

It's tough -- if someone is hitting you at > 1gbps, that exceeds a gigabit ethernet port -- typically on shared services you only pay for that much. Any more than that and it affects all the other customers on the shared network. So yes, Rackspace has the bandwidth for it, but at the same time if you were Rackspace, would you let our DDoS take down the other hosts in the datacenter?

Gigenet / Black Lotus / Prolexic comes in and says -- we have excess bandwidth, we'll take care of it. Then they proxy clean traffic to you. Unfortunately as we are learning, it's also very expensive.

Re: Posterous is being DDoS'd

#29

We're getting about 500k packets per second, 500mbps to 1.5gbps peak, it's a synflood from a botnet. Typically we can IP hop and null-route the old IP's. That usually buys us about a day until the botnet phones home to get the new IP's, at which point we just hop again. Since our DNS TTL is only 5 minutes at most we are down 5 minutes. TODAY, the attackers hopped IP's to our new IP immediately. So they appear to be l…

Do all the caching name servers respect the TTL though?

They better. So far so good.

Re: Posterous is being DDoS'd

#30

I seriously doubt this is the reason but they insulted a lot of people with their "switch to us" targeted attacks. That was classless.

It wasn't classless, it was marketing. No one wants their competitors poaching their clients but in business you deal with it. That said, the same thought went through my mind, I wonder if it's related.

That sounds very clear and professional. Unfortunately, it doesn't always work out that way.

Most of their competitors are in the same situation they are in.

Being the first one to torpedo the other ship and not expecting someone to get upset or God forbid do something awful in return is naive.

I think there is a difference in poaching your competitor's customers and claiming they are "no good".

Post reply on HN