Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“We have been experiencing a catastrophic DDoS attack”

status.linode.com

Post reply on HN
Print view View original

131–140 of 151 posts

Re: “We have been experiencing a catastrophic DDoS attack”

#131

Post by lmz »

technion wrote: post #72
throwsep3 wrote: post #36

I wonder if this is a diversion to keep Linode's security team busy so they won't notice someone compromising the Xen nodes with XSA-185/6/7/8?

I've noted the AWS security bulletins[0] list nearly every Xen advisory with "AWS customers' data and instances are not affected by these issues, and there is no customer action required." It would appear? you'd need to go back for quite a few months of being unpatched to find a genuine issue. Unless something about Amazon's mitigations don't apply universally. [0] https://aws.amazon.com/security/security-bulletins/

AWS (and other large hosts: https://www.xenproject.org/security-policy.html search for predisclosure) get notified before the public.

Re: “We have been experiencing a catastrophic DDoS attack”

#132

Post by naveen99 »

SEJeff wrote: post #45
chrischen wrote: post #40

Does the US have a competent cyber-crime division that can handle stuff like this?

Yes, the FBI. They're fantastic at it and part of their job is helping businesses recover from compromise and going after the attackers. However, they're overworked government employees with not enough resources.

> overworked government employees

An oxymoron ?

Re: “We have been experiencing a catastrophic DDoS attack”

#133

Post by jonahx »

viraptor wrote: post #57
jonahx wrote: post #34

Can anyone recommend a good article that explains how attacks like these work, and what is required to stop them? Also, we're on Heroku and they advertise Ddos mitigation as a feature, but "mitigation" sounds non-commital and I'm curious how they'd fare against a similar attack?

It's non-commital, because at some point, when you have enough zombie hosts properly distributed all over the world attacking you, your only defence is - have more bandwidth than the attackers. If your peers can't filter out the traffic before it hits your network and it simply saturates your pipes, there's nothing you can do inside the company anymore.

Thanks for the reply. Can you elaborate on what you mean by "peers" in the above? Eg, who (or what) would Heroku's peers be?

Re: “We have been experiencing a catastrophic DDoS attack”

#134

Post by exhilaration »

technion wrote: post #72
throwsep3 wrote: post #36

I wonder if this is a diversion to keep Linode's security team busy so they won't notice someone compromising the Xen nodes with XSA-185/6/7/8?

I've noted the AWS security bulletins[0] list nearly every Xen advisory with "AWS customers' data and instances are not affected by these issues, and there is no customer action required." It would appear? you'd need to go back for quite a few months of being unpatched to find a genuine issue. Unless something about Amazon's mitigations don't apply universally. [0] https://aws.amazon.com/security/security-bulletins/

Amazon and other big cloud providers get xen security fixes first, there was an HN discussion about it. Some they've probably already implemented the fix.

Re: “We have been experiencing a catastrophic DDoS attack”

#135

Post by boulos »

xmatos wrote: post #110

I don't get the hate towards linode here, on hacker news. I've been their client for a couple of years now and I find it an excellent vps provider. Excellent uptime and performance at a pretty good price. AWS has a few outages every year. Google just had one last week. Azure sucks balls. So, why the hate? Is it because it competes with some ycombinator startups?

For us (Google), I think you're referring to: https://status.cloud.google.com/incident/compute/16017

which was caused by our own maintenance several weeks apart (the root cause description is really quite good).

I think the distinction people make implicitly is a 25 minute outage versus 8 hours. DDoS attacks suck, but they're just standard these days. As a customer though, any source of (network) outage usually has the same outcome: "my site is unreachable (and I don't care why)".

The reason we (and AWS and others) offer multiple datacenters/zones within a Linode is good at what they do, but any customer in Atlanta just had to wait this entire event out.

Disclosure: I work on Compute Engine.

Re: “We have been experiencing a catastrophic DDoS attack”

#136

Post by theyak »

noir-york wrote: post #87
mappu wrote: post #12

Earlier quoted context omitted.

We stuck with Linode after that, but the important parts now failover to Vultr.

We stuck with Linode after last Christmas too, but this new attack looks like the final straw. How have you found Vultr?

We've moved off vultr. Too many network down times for no known reason and hard reboots on our servers causing loss of data. A friend has had similar experiences. Their support was also poor.

Re: “We have been experiencing a catastrophic DDoS attack”

#137

Post by warbiscuit »

mratzloff wrote: post #130
hyperpape wrote: post #103

Earlier quoted context omitted.

What in that article makes you think that? I don't see it. They do say "we'll have to upgrade Xen nodes", but they don't mention the DDoS or link them.

Why would they? Timing and the fact that it has happened before make it seem likely.

The XEN update was a scheduled thing -- I got an email about it weeks ago (had one linode I hadn't moved to KVM), and it was already scheduled for this weekend.

That said, I don't disagree that the attackers might be trying to distract the team while they exploit that... though I don't see what it gets them compared to quietly exploiting the XEN issues before they were common knowledge on 9/8.

Re: “We have been experiencing a catastrophic DDoS attack”

#138

Post by warbiscuit »

fizzbatter wrote: post #122
xmatos wrote: post #110

I don't get the hate towards linode here, on hacker news. I've been their client for a couple of years now and I find it an excellent vps provider. Excellent uptime and performance at a pretty good price. AWS has a few outages every year. Google just had one last week. Azure sucks balls. So, why the hate? Is it because it competes with some ycombinator startups?

I don't follow this too closely, so this is just wild speculation from me: But could it simply be severity of the attacks? I keep seeing comments about a 2 week ddos attack last christmas - that's something that i would be shocked to see Google/AWS succumb to. Not that Google/AWS attacks don't happen, i just can't imagine them being down for ~2weeks (I imagine it was just one datacenter from Linode, not the entire se…

They weren't down for the entire two weeks, but various datacenters went up and down for hours, then quieted down for a few days, then was back again, then another hit; stretching across two weeks.

One thing that took them so long was that their upstream ISPs at some of the datacenters were themselves unable to handle the DDOS, so they had to switch ISPs, which took a while.

I don't see Google/AWS as easy to attack; but I'm not sure why similar tier players like DigitalOcean aren't being hit -- or maybe they're just less transparent about things, or are actually a smaller target (didn't think they were?).

edit: here's a postmortem from linode of the christmas attack - https://blog.linode.com/2016/01/29/christmas-ddos-retrospect...

Re: “We have been experiencing a catastrophic DDoS attack”

#139

Post by mikebutash »

constit-protec wrote: post #82

Godaddy has been getting attacked a lot recently as well. Who is likely behind attacking servers, whether other server companies or governments?

When you are the dns root record and in many cases hosting too for some 60-70 million domains, someone in the internet is wanting to attack someone at godaddy all the time. There isn't a time they aren't being attacked somehow in all reality, and I'd presume it's much the same for Linode.

The means of really combating a ddos is costly and extensive, this is why most use a service like prolexic or silverline, and typically with some massive infrastructures that comes with it. Anything less than n-by 40gb disposable internet pipes, preferably regional as you are, you can/will be smited at will.

Re: “We have been experiencing a catastrophic DDoS attack”

#140

Post by hyperpape »

mratzloff wrote: post #130
hyperpape wrote: post #103

Earlier quoted context omitted.

What in that article makes you think that? I don't see it. They do say "we'll have to upgrade Xen nodes", but they don't mention the DDoS or link them.

Why would they? Timing and the fact that it has happened before make it seem likely.

You're the one who linked to the piece. I think it's your job to be clear that it doesn't say that, it's your conclusion.
Post reply on HN