Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do you handle DDoS attacks?

news.ycombinator.com

Post reply on HN
Print view View original

91–100 of 114 posts

Re: Ask HN: How do you handle DDoS attacks?

#91

Post by damm »

1. Have a big enough pipe; if you are getting a DDoS attack of 2Gigabits/second and your uplink is 1Gigabit there is nothing you can do except look for someone else to filter your traffic. (They have to basically take on the 2gig ddos; filter it and then pass back the valid traffic to you).

Verisign and others offer this service; typically using DNS. However often they support BGP

2. Add limiting factors; if you have an abusive customer rate limit them in nginx. If you are expecting a heavy day rate limit the whole site.

3. Stress testing and likely designing your website to withstand DDoS attacks.

You can cache or not cache; that's not really the question. Handling a DDoS means what can you do to mitigate the extreme amount of traffic and still allow everything else to work.

Re: Ask HN: How do you handle DDoS attacks?

#92

Post by anotherdpk »

ryanlol wrote: post #73
wesleytodd wrote: post #25

My startup's site gets DDOS'd about once a week. We have seen a huge range of attacks from UDP floods, to wordpress pingback attacks, to directed attacks on our services. We have many layers of protection: * We run iptables and an api we wrote on our ingest servers. We run failtoban on a separate set of servers. When fail2ban sees something, we have it hit the api and add the iptables rules. This offloads the cpu of…

Cloudflare doesn't help you against UDP floods if your backend is publicly accessible on the internet. For example: curl https://104.154.116.193 -H 'Host: www.stream.me' -v -k

Indeed, it's a recent integration and we haven't yet shut off that IP.

Btw, check out curl's --resolve flag. You can use it to override default DNS resolution and can then drop the -k flag.

Re: Ask HN: How do you handle DDoS attacks?

#94

Post by ryanlol »

samplonius wrote: post #89
jaypaulynice wrote: post #69

Earlier quoted context omitted.

Have you heard about cache busting? Someone just needs to request a page that's not cached and the request will always hit your web servers.

If there are 90GB of static files, and 60GB are in the Varnish cache, cache busting will be pretty ineffective.

If there's any dynamic content and the request hits that, Varnish cache will be pretty ineffective.

Re: Ask HN: How do you handle DDoS attacks?

#95

Post by voltagex_ »

Kenji wrote: post #90
buro9 wrote: post #38

I've faced DoS attacks for years as I run internet forums. The simple advice for layer 7 (application) attacks: 1. Design your web app to be incredibly cacheable 2. Use your CDN to cache everything 3. When under attack seek to identify the site (if you host more than one) and page that is being attacked. Force cache it via your CDN of choice. 4. If you cannot cache the page then move it. 5. If you cannot cache or mov…

Uh, stupid question but how do you cache a website like for example this comment thread on hackernews? Suppose a DDoSer calls this comment thread a lot of times. The request has to go through to the server because when I hit F5 or post a comment myself, I see the comments in realtime. How do you handle that exactly? Does caching for a few seconds help already, or does the backbone push updated sites to the CDN server…

What you can do with a site like HN will be different than if you're a shopping getting DDoSed on Black Friday by a competitor.

You can put the whole of HN into read only mode if needed and it'll have no real impact; disallowing purchases on MyAmazonCompetitor.com would be catastrophic.

Re: Ask HN: How do you handle DDoS attacks?

#96

Post by voltagex_ »

Most of the responses here deal with bandwidth floods. Is that really the most common DDoS?

Thinking like an attacker, wouldn't the most effective DoS be to find a CPU or memory intensive part of an application and use a small amount of bandwidth to create a large impact?

Re: Ask HN: How do you handle DDoS attacks?

#97

Post by TimMeade »

damm wrote: post #91

1. Have a big enough pipe; if you are getting a DDoS attack of 2Gigabits/second and your uplink is 1Gigabit there is nothing you can do except look for someone else to filter your traffic. (They have to basically take on the 2gig ddos; filter it and then pass back the valid traffic to you). Verisign and others offer this service; typically using DNS. However often they support BGP 2. Add limiting factors; if you have…

We got hit by one about a month ago that was over 20Gb. Even a 10Gb pipe has limits.

Re: Ask HN: How do you handle DDoS attacks?

#98

Post by owlish »

Kenji wrote: post #90
buro9 wrote: post #38

I've faced DoS attacks for years as I run internet forums. The simple advice for layer 7 (application) attacks: 1. Design your web app to be incredibly cacheable 2. Use your CDN to cache everything 3. When under attack seek to identify the site (if you host more than one) and page that is being attacked. Force cache it via your CDN of choice. 4. If you cannot cache the page then move it. 5. If you cannot cache or mov…

Uh, stupid question but how do you cache a website like for example this comment thread on hackernews? Suppose a DDoSer calls this comment thread a lot of times. The request has to go through to the server because when I hit F5 or post a comment myself, I see the comments in realtime. How do you handle that exactly? Does caching for a few seconds help already, or does the backbone push updated sites to the CDN server…

If you're getting more traffic than 1 request/s, it's less work to generate a static cached version on the cadence of ~1 second than to dynamically generate the content for each request.

Re: Ask HN: How do you handle DDoS attacks?

#99

Post by TimMeade »

damm wrote: post #91

1. Have a big enough pipe; if you are getting a DDoS attack of 2Gigabits/second and your uplink is 1Gigabit there is nothing you can do except look for someone else to filter your traffic. (They have to basically take on the 2gig ddos; filter it and then pass back the valid traffic to you). Verisign and others offer this service; typically using DNS. However often they support BGP 2. Add limiting factors; if you have…

We got hit by one about a month ago that was over 20Gb. Even a 10Gb pipe has limits.

Re: Ask HN: How do you handle DDoS attacks?

#100

Post by jdcarr »

Kenji wrote: post #90
buro9 wrote: post #38

I've faced DoS attacks for years as I run internet forums. The simple advice for layer 7 (application) attacks: 1. Design your web app to be incredibly cacheable 2. Use your CDN to cache everything 3. When under attack seek to identify the site (if you host more than one) and page that is being attacked. Force cache it via your CDN of choice. 4. If you cannot cache the page then move it. 5. If you cannot cache or mov…

Uh, stupid question but how do you cache a website like for example this comment thread on hackernews? Suppose a DDoSer calls this comment thread a lot of times. The request has to go through to the server because when I hit F5 or post a comment myself, I see the comments in realtime. How do you handle that exactly? Does caching for a few seconds help already, or does the backbone push updated sites to the CDN server…

The important thing you need to assess is how critical is it that clients receive fresh data.

You can imagine that for a real time service it would be better to provide a timeout immediately rather than providing stale data.

HN is an example of a near on-line site where some delay is perfectly acceptable. No one cares that they're receiving a 2 second old page, it's better for the site users to reveive old data fast rather than new data slow.

If you use nginx the following commands would help out significantly (if I remember them correctly)

proxy_cache_use_stale updating proxy_cache_lock on proxy_cache_lock_timeout 1s

This config allows nginx to fetch cache updates while serving clients and when fresh data is received from the upstream application server it'll use that immediately.

If that's wrong hopefully someone can correct the conf.

Post reply on HN