Live data from Hacker News

“We have been experiencing a catastrophic DDoS attack”

status.linode.com

71–80 of 151 posts

Re: “We have been experiencing a catastrophic DDoS attack”

#71
post #25

Earlier quoted context omitted.

There are a handful of environments that can sustain a large, coordinated DDOS attack. Can you sink 10-20Gb/s of traffic forever? Not cost effectively.

If you live in 2002, nah. In 2016? Yes. Bandwidth is cheap and linode has high margins.

Bandwidth is cheap but attacks are cheap too.

Re: “We have been experiencing a catastrophic DDoS attack”

#72

I wonder if this is a diversion to keep Linode's security team busy so they won't notice someone compromising the Xen nodes with XSA-185/6/7/8?

I've noted the AWS security bulletins[0] list nearly every Xen advisory with "AWS customers' data and instances are not affected by these issues, and there is no customer action required."

It would appear? you'd need to go back for quite a few months of being unpatched to find a genuine issue. Unless something about Amazon's mitigations don't apply universally.

[0] https://aws.amazon.com/security/security-bulletins/

Re: “We have been experiencing a catastrophic DDoS attack”

#73

Earlier quoted context omitted.

It's pretty much the same. You almost never want writes on both sides (now in a failover plan anyway), so as long as you have a switch for which side receives the writes, it's simple.

Site A, insure any writes use odd numbers in unique field. Site B, insure any writes use even numbers for unique field. This avoids conflict.

Or allow writes in both datacenters with randomized tokens as keys. If you need datacenter-affinity for certain events, use one of the token bytes to encode the author datacenter. Updates that don't have to land in order can be written in an eventually consistent manner. Write a feed of changes in each datacenter and have the peers consume this update feed. Viola, partition-tolerant master-master with failover.

Re: “We have been experiencing a catastrophic DDoS attack”

#74
post #32

Earlier quoted context omitted.

Infrastructure to service requests is not (comparitively). Rarely is it trivial to distinguish DDoS traffic from legitimate traffic.

Usually it is and according to linode it is so in this case too. Edit: Not sure what alternate reality the downvoter lives in, but vast majority of the attacks these days are just "dumb" packet floods or even easier to filter reflection attacks. (Linode clarified this to be a mix of dns and ntp traffic on IRC) But hey, go on and find me a layer 7 attack that'll take down entire datacenters :)

One difference between Linode and providers like AWS is that the typical deployment architecture on Linode still exposes customer VPSes to direct L3 internet traffic whereas on a best-practices AWS deployment that is almost never the case.

I'd imagine it's easier to filter out bogus L3 traffic when the vast majority of your target IP space comes with explicit configuration as to what sort of L7 application traffic is acceptable.

Re: “We have been experiencing a catastrophic DDoS attack”

#76

I wonder if this is a diversion to keep Linode's security team busy so they won't notice someone compromising the Xen nodes with XSA-185/6/7/8?

Apparently they came to the same conclusion—just a bit late.

https://status.linode.com/incidents/frq9sz7dyb1x

Re: “We have been experiencing a catastrophic DDoS attack”

#77
post #32

Earlier quoted context omitted.

Infrastructure to service requests is not (comparitively). Rarely is it trivial to distinguish DDoS traffic from legitimate traffic.

Usually it is and according to linode it is so in this case too. Edit: Not sure what alternate reality the downvoter lives in, but vast majority of the attacks these days are just "dumb" packet floods or even easier to filter reflection attacks. (Linode clarified this to be a mix of dns and ntp traffic on IRC) But hey, go on and find me a layer 7 attack that'll take down entire datacenters :)

Stolen credit cards are cheap, and they buy a lot of GCE/AWS resources.

source: been on the receiving end of a several hundred gigabit L7/HTTP DDoS from the aforementioned providers.

Re: “We have been experiencing a catastrophic DDoS attack”

#78

Earlier quoted context omitted.

It's pretty much the same. You almost never want writes on both sides (now in a failover plan anyway), so as long as you have a switch for which side receives the writes, it's simple.

Site A, insure any writes use odd numbers in unique field. Site B, insure any writes use even numbers for unique field. This avoids conflict.

I do the same with tree nodes, works extremely well. Two active masters and one slave configured as master, conflicts are non-existant since I found out about this "trick".

Re: “We have been experiencing a catastrophic DDoS attack”

#79

Earlier quoted context omitted.

For our setup we just have master-master replication across two different hosting providers.

I've been wondering about this lately. Is it really feasible for a small (one man?) team to keep master-master MySQL replication over WAN running smoothly?

If it's as easy and reliable as it is with Postgres (and I'd suspect it is), the answer is in my experience yes.

Re: “We have been experiencing a catastrophic DDoS attack”

#80

Earlier quoted context omitted.

For our setup we just have master-master replication across two different hosting providers.

I've been wondering about this lately. Is it really feasible for a small (one man?) team to keep master-master MySQL replication over WAN running smoothly?

I set it up once like three years ago and it just works.

I switch between either master with just a DNS update that propagates within 5 minutes.

Post reply on HN