Live data from Hacker News

Ask HN: How is DDoS protection implemented?

news.ycombinator.com

1–10 of 58 posts

Ask HN: How is DDoS protection implemented?

#1
The big services (Google, Cloudflare, etc.) provide DDoS attack mitigation (and seem to succeed), but details on their tactics are rare (at least I did not find in-depth information on that).

I guess to make this work well you have to do classification (regular request vs. malicious) on several protocol layers and then reroute or drop packets accordingly. But how does that prevent severe service degradation - you still have to do some kind of work (in computation and energy) on the listening side or can fat edge-servers just eat that up?

Re: Ask HN: How is DDoS protection implemented?

#2
Not an expert but I would guess at least the following: traffic filtering, peer traffic filtering by (possibly dynamic and automated) agreement, traffic classification and anomaly detection (DNS/TCP/HTTP(S)/etc.), routing different clients (based on origin AS and/or geolocation) to different IPs through DNS, hosted web frontends, web-level active user challenges, potentially dynamically altering the advertisement of routes, and by charging so much money the moment you need to use them that buying extra bandwidth and netblocks isn't an issue for them. Probably some of them also drop to high-overhead traffic reduction modes which can expand frontend IPs and DNS response segmentation, dropping DNS TTLs and spinning up new proxy systems in order to better filter out robotic attackers. Many also probably create/profile/buy various browser fingerprinting techniques and may have a library of non publicly disclosed approaches available for additional mitigation during high bandwidth attacks. Oh yeah, and replicating a static cache as a cheap means of degraded service provisioning.

Re: Ask HN: How is DDoS protection implemented?

#3

Not an expert but I would guess at least the following: traffic filtering, peer traffic filtering by (possibly dynamic and automated) agreement, traffic classification and anomaly detection (DNS/TCP/HTTP(S)/etc.), routing different clients (based on origin AS and/or geolocation) to different IPs through DNS, hosted web frontends, web-level active user challenges, potentially dynamically altering the advertisement of…

[flagged]

Re: Ask HN: How is DDoS protection implemented?

#4
One way these companies mitigate DDoS attacks is by being huge. If you have a small house w/ one entrance, there's no great way to manage 1000 people trying to get through the front door. If you have a huge house w/ dozens of entrances, dealing with 1000 people trying to get in the building is much more manageable :)

From https://en.wikipedia.org/wiki/DDoS_mitigation:

One technique is to pass network traffic addressed to a potential target network through high-capacity networks with "traffic scrubbing" filters.

Re: Ask HN: How is DDoS protection implemented?

#5
I was lead developer on Arbor Network's DDoS product in the early 2000s (I left in 2005 to start Matasano Security). My information on this is surely dated, but people seem to still be using the same terminology now as then.

You can break down DDoS into roughly three categories:

1. Volumetric (brute force)

2. Application (targeting specific app endpoints)

3. Protocol (exploiting protocol vulnerabilities)

DDoS mitigation providers concentrate on 1 & 3.

The basic idea is: attempt to characterize the malicious traffic if you can, and or divert all traffic for the target. Send the diverted traffic to a regional "scrubbing center"; dirty traffic in, clean traffic out.

The scrubbing centers buy or build mitigation boxes that take large volumes of traffic in and then do heuristic checks (liveness of sender, protocol anomalies, special queueing) before passing it to the target. There's some in-line layer 7 filtering happening, and there's continuous source characterization happening to basic network layer filters back towards ingress.

You can do pretty simple statistical anomaly models and get pretty far with attacker source classification, and to track targets and be selective about what things need to be diverted.

A lot of major volumetric attacks are, at the network layer, pretty unsophisticated; they're things like memcached or NTP floods. When you're special-casing traffic to a particular target through a scrubbing center, it's pretty easy to strip that kind of stuff off.

Re: Ask HN: How is DDoS protection implemented?

#6
The easy answer: Load balancing

Anycast is the most important piece of the puzzle, allowing you to route traffic to a bunch of different locations.

Let's say you can handle 10 Gbps at a single location. If the traffic is evenly split between 100 destinations then you can have a single IP that can handle 1 Tbps of traffic.

Of course, the setup behind these IPs might vary a lot, and one might even use DNS load balancing in front of the IPs.

Re: Ask HN: How is DDoS protection implemented?

#7
By far the biggest part of attack mitigation in my experience is out-scaling the attack. A well written and configured application stack can handle a decent amount of traffic itself before becoming bogged down processing malicious traffic, but at some point you'll cap out either the application, the NIC, the upstream switch, the router, or the ISP line, if your application is running in just one place. To get around that, huge providers like the ones you listed are heavily multi-homed. This means they announce their traffic routes to the internet from multiple locations, so traffic naturally flows to the closest (hops wise, not necessarily geographically speaking) endpoint.

From there, you can add layers of protection ranging from simple things like blocking traffic that is obviously malicious (TCP flags, port numbers, etc) to more complex things like pattern recognition in both the overall trends of the data and on a per-packet basis. After you've decided with a decent certainty that it's not malicious traffic, you pass it off to the actual backend service.

For systems that are designed to scale horizontally, that may be a neighboring machine (or even the same machine) in that data center. For single-homed backend systems that can't scale horizontally to multiple locations, that "clean" traffic is then sent via some mechanism (possibly a GRE tunnel, possibly just raw internet traffic to a secret IP) to the backend service. Depending on the methodology used, the filtering may be a true bidirectional proxy, in which case the reply goes back to the scrubber and then out to the original sender, or it may be a unidirectional proxy, in which case the reply goes directly back to the original sender.

All attack mitigation works in some way like this, whether it be by designing your application from the beginning to be multi-homed and able to run in multiple datacenters, or by installing a separate mitigation layer that scrubs attack traffic.

Re: Ask HN: How is DDoS protection implemented?

#8
From my personal low-end server perspective (which has stood up to simple attacks from Russian IPs), I have the following:

1. Static page caching (in RAM ideally) - dynamically generated content will kill you quicker than anything else, especially calls to a database. WordPress is very easy to kill in it's default state.

2. Kill high frequency requests from the same location as quickly as possible (make sure your response is less than the data they send you - ultimately you want their systems to be busier than yours). You want to free the port up as quickly as possible.

3. Move anybody you can identify as a legitimate user (credentials, low frequency requests) out to another server if possible.

Firewall wise, my system sits on the cloud, so usually high frequency traffic is the only issue I have to deal with. Interested to hear any advice of other people here.

Re: Ask HN: How is DDoS protection implemented?

#10

The easy answer: Load balancing Anycast is the most important piece of the puzzle, allowing you to route traffic to a bunch of different locations. Let's say you can handle 10 Gbps at a single location. If the traffic is evenly split between 100 destinations then you can have a single IP that can handle 1 Tbps of traffic. Of course, the setup behind these IPs might vary a lot, and one might even use DNS load balancin…

Load balancing is in place for all but the most trivial sites, though, so what you're really saying is horizontal scaling. Which is fine but expensive compared to pattern based mitigation techniques.
Post reply on HN