Live data from Hacker News

Ask HN: How do you handle DDoS attacks?

news.ycombinator.com

101–110 of 114 posts

Re: Ask HN: How do you handle DDoS attacks?

#101
post #90
post #38

I've faced DoS attacks for years as I run internet forums. The simple advice for layer 7 (application) attacks: 1. Design your web app to be incredibly cacheable 2. Use your CDN to cache everything 3. When under attack seek to identify the site (if you host more than one) and page that is being attacked. Force cache it via your CDN of choice. 4. If you cannot cache the page then move it. 5. If you cannot cache or mov…

Uh, stupid question but how do you cache a website like for example this comment thread on hackernews? Suppose a DDoSer calls this comment thread a lot of times. The request has to go through to the server because when I hit F5 or post a comment myself, I see the comments in realtime. How do you handle that exactly? Does caching for a few seconds help already, or does the backbone push updated sites to the CDN server…

Make your comments system a static site generator, so that each comment generates a static HTML page and you serve that statically. 4chan does this.

Re: Ask HN: How do you handle DDoS attacks?

#102

Most of the responses here deal with bandwidth floods. Is that really the most common DDoS? Thinking like an attacker, wouldn't the most effective DoS be to find a CPU or memory intensive part of an application and use a small amount of bandwidth to create a large impact?

At AWS re:Invent 2015, Amazon claimed that 15% of attacks were at layer 7, 65% were network level bandwidth floods, and 20% were network level state exhaustion [1].

[1]: https://youtu.be/Ys0gG1koqJA?t=229

Re: Ask HN: How do you handle DDoS attacks?

#104

Most of the responses here deal with bandwidth floods. Is that really the most common DDoS? Thinking like an attacker, wouldn't the most effective DoS be to find a CPU or memory intensive part of an application and use a small amount of bandwidth to create a large impact?

Attacks that are heavy on L1-4 are the hardest to protect against because of the need for large fixed infrastructure (peering/transit).

L7 attacks can be scrubbed by the same infrastructure. Beyond that, it's all a matter of detection. The computational expense of L7 inspection can be mitigated by sampling or scaled with ECMP. You may see a "WAF" (Web Application Firewall) enter the picture at this level.

Re: Ask HN: How do you handle DDoS attacks?

#106

I use and recommend hosting with OVH if you are worried about DDOS and serving a Western market. No affiliation, just a happy customer. OVH include DDOS protection by default[0] and they have a very robust backbone network[1] in Europe and North America that they own and operate themselves (this is how & why anti-DDOS is standard with them). For quick side-projects I still fire up a DigitalOcean instance or two becau…

Tangential, but how do you find OVH? Their hardware, bandwidth, uptime, customer service? I ask because of the conflicting reviews of OVH that a quick google search reveals.

At work we're using OVH for our production, we've been with them for several years. The key point is that the price-performance ratio is very difficult to beat, and it offsets the problems we've had.

We've had very few hardware-related issues, a disk failing or a motherboard to be replaced. In all of those cases, the component were swapped promptly and we've been kept informed of the progress.

Where we're unhappy is with the network, especially with their vRack offering. Looking back at our production incidents of the past 6 months, about 50% of them were caused by some vRack problem where at the same time the public interfaces were up and running just fine.

We're generally happy with customer service, but we pay for VIP support and we speak French to OVH's support agents (I believe that the latter helps a lot).

Re: Ask HN: How do you handle DDoS attacks?

#107
post #90
post #38

I've faced DoS attacks for years as I run internet forums. The simple advice for layer 7 (application) attacks: 1. Design your web app to be incredibly cacheable 2. Use your CDN to cache everything 3. When under attack seek to identify the site (if you host more than one) and page that is being attacked. Force cache it via your CDN of choice. 4. If you cannot cache the page then move it. 5. If you cannot cache or mov…

Uh, stupid question but how do you cache a website like for example this comment thread on hackernews? Suppose a DDoSer calls this comment thread a lot of times. The request has to go through to the server because when I hit F5 or post a comment myself, I see the comments in realtime. How do you handle that exactly? Does caching for a few seconds help already, or does the backbone push updated sites to the CDN server…

Literally only cache for 1 or 2 seconds at a time.

Lots of people use page caching to speed up their website, but that's a mistake, since caching means stale data on dynamic sites. Caching should only be used to solve resource issues, not latency issues.

Your entire site should be fast already without caching. This comments page should only take a few milliseconds to generate. If it doesn't, then something's wrong with the database queries.

I will never understand how some sites take hundreds of milliseconds to generate a page.

Re: Ask HN: How do you handle DDoS attacks?

#108
post #90
post #38

I've faced DoS attacks for years as I run internet forums. The simple advice for layer 7 (application) attacks: 1. Design your web app to be incredibly cacheable 2. Use your CDN to cache everything 3. When under attack seek to identify the site (if you host more than one) and page that is being attacked. Force cache it via your CDN of choice. 4. If you cannot cache the page then move it. 5. If you cannot cache or mov…

Uh, stupid question but how do you cache a website like for example this comment thread on hackernews? Suppose a DDoSer calls this comment thread a lot of times. The request has to go through to the server because when I hit F5 or post a comment myself, I see the comments in realtime. How do you handle that exactly? Does caching for a few seconds help already, or does the backbone push updated sites to the CDN server…

Cache everything a guest accesses for 5 minutes or more. Vary on the specific cookie that represents a signed-in user.

None of my guests have noticed this, and it has increased most of my analytics numbers as my pages are faster too.

The signed-in users, they get the dynamic pages.

But now the cookie that identifies the user is what you use to correlate any attack traffic, the attacker is forced to (somewhat) identify themselves and you can then revoke their authentication status or ban the account.

Finally you captcha and/or rate-limit the login page.

This is effectively what I do on my sites, the pages themselves and the underlying API all cache if the cookie or access token is absent.

This is trivial to do within the code, but can be harder to do with the CDN/security layer (who need to support a "vary on cookie" or "bypass cache on cookie" or equivalent).

Re: Ask HN: How do you handle DDoS attacks?

#109
post #94

Earlier quoted context omitted.

If there are 90GB of static files, and 60GB are in the Varnish cache, cache busting will be pretty ineffective.

If there's any dynamic content and the request hits that, Varnish cache will be pretty ineffective.

Actually Varnish is great here, one normalises the requests and retains only the querystrings that are valid for your application filtering out (removing) all those that are not valid.

The key thing is, you know your application, and you know what the valid keys are and the valid value ranges. If you can express that in your HTTP server and discard requests then it can be very cheaply done.

A forum really doesn't have that many distinct URLs, and so this is easily done. It would be harder on a much more complex application, but the original question related to these smaller side-project applications.

Re: Ask HN: How do you handle DDoS attacks?

#110

Earlier quoted context omitted.

Tangential, but how do you find OVH? Their hardware, bandwidth, uptime, customer service? I ask because of the conflicting reviews of OVH that a quick google search reveals.

I found OVH's offer to be very good on every point, except customer service. I'm mostly using Kimsufi dedicated servers, and let's just say that when shit goes wrong, you're left alone in the dark. Anecdote: I had my dedicated server suddenly go down because it overheated. Wouldn't come back to life. Two days after submitting a ticket and getting no input, the machine suddenly came back up without any explanation. A…

This is because kimsufi are the cheapest dedicated server of OVH. They do provide support but its delayed and restricted to hardware issues. These servers are intendent for playing around and testing, hence the low price. The offer is also minimal. I use it for some toy web site hosting and mail hosting. It's good for boostraping.
Post reply on HN