Live data from Hacker News

Ask HN: How do you handle DDoS attacks?

news.ycombinator.com

31–40 of 114 posts

Re: Ask HN: How do you handle DDoS attacks?

#31

My startup's site gets DDOS'd about once a week. We have seen a huge range of attacks from UDP floods, to wordpress pingback attacks, to directed attacks on our services. We have many layers of protection: * We run iptables and an api we wrote on our ingest servers. We run failtoban on a separate set of servers. When fail2ban sees something, we have it hit the api and add the iptables rules. This offloads the cpu of…

One other thing to add: If they attackers are persistent, there is really no way to guarantee zero down time. THEY WILL FIND A WAY. Just make sure your stake holders know you are doing everything in your power to resolve the issues, and then actually do those things. An anecdote: We had been seeing DDOS attacks for a few weeks, so we had most everything locked down and working. But then suddenly one of the most impor…

Great tips and examples. Makes me wonder what site/service you work on that attracts attackers like that.

Re: Ask HN: How do you handle DDoS attacks?

#33

Earlier quoted context omitted.

One other thing to add: If they attackers are persistent, there is really no way to guarantee zero down time. THEY WILL FIND A WAY. Just make sure your stake holders know you are doing everything in your power to resolve the issues, and then actually do those things. An anecdote: We had been seeing DDOS attacks for a few weeks, so we had most everything locked down and working. But then suddenly one of the most impor…

Great tips and examples. Makes me wonder what site/service you work on that attracts attackers like that.

https://www.stream.me/

We have some controversial users...

Re: Ask HN: How do you handle DDoS attacks?

#34
post #2

https://www.cloudflare.com

You mean the biggest MiTM on the web?[0]

The only reason why they're not constantly called out by serious infosec folk for their scam is because they hire guys also involved in DefCon/BlackHat planning (try to sneak a hostile talk against Cloudflare past REDACTED[2] who btw is also advising Mr. Robot). It's lobbying at its finest.

[0] https://scotthelme.co.uk/tls-conundrum-and-leaving-cloudflar...

[1] https://blog.torproject.org/blog/trouble-cloudflare

EDIT: [2] redacted name since there is more than one, please duckduckgo by yourself.

Re: Ask HN: How do you handle DDoS attacks?

#35

Earlier quoted context omitted.

Great tips and examples. Makes me wonder what site/service you work on that attracts attackers like that.

https://www.stream.me/ We have some controversial users...

I guess some people just really hate kittens :)

https://www.stream.me/kittendorm

Re: Ask HN: How do you handle DDoS attacks?

#36

I use and recommend hosting with OVH if you are worried about DDOS and serving a Western market. No affiliation, just a happy customer. OVH include DDOS protection by default[0] and they have a very robust backbone network[1] in Europe and North America that they own and operate themselves (this is how & why anti-DDOS is standard with them). For quick side-projects I still fire up a DigitalOcean instance or two becau…

Have you been DDoSed while on OVH? I've heard there are a bunch of providers who claim DDoS prevention, but what that means in practice is just "we'll take your site offline right away and not charge you for the incoming bandwidth!". Super helpful.

I've had several attacks hit my services on OVH. Not once did they null route, and as far as I could tell the majority of attack traffic was filtered with only a minor service interruption before mitigation kicked in. Granted these were shitty $10 booter services hitting me and not a "real" DDoS.

Be careful using services like game servers or VOIP or anything else using UDP though, since UDP is subject to much more stringent filtering at OVH and may get affected during mitigation.

Re: Ask HN: How do you handle DDoS attacks?

#37
Disclaimer: I work for a hosting company, but these views are my own personal opinions which I held even before working where I currently do.

This is one of the reasons I would consider managed hosting as opposed to AWS, Digital Ocean, etc. With any good managed hosting provider, they are going to take steps to help deal with the DDoS. Depending on your level of service and the level of the attack, of course. But they will have an interest in helping you deal with and mitigate the attacks.

The reality is that true DDoS solutions are expensive, and if you have a "small website" then you're probably not going to be able to afford them. But if you're at a good sized hosting provider, they're going to need to have these solutions themselves and can hopefully put them to use to protect your site.

Re: Ask HN: How do you handle DDoS attacks?

#38
I've faced DoS attacks for years as I run internet forums.

The simple advice for layer 7 (application) attacks:

1. Design your web app to be incredibly cacheable

2. Use your CDN to cache everything

3. When under attack seek to identify the site (if you host more than one) and page that is being attacked. Force cache it via your CDN of choice.

4. If you cannot cache the page then move it.

5. If you cannot cache or move it, then have your CDN/security layer of choice issue a captcha challenge or similar.

The simple advice for layer 3 (network) attacks:

1. Rely on the security layer of choice, if it's not working change vendor.

On the L3 stuff, when it comes to DNS I've had some bad experiences (Linode, oh they suffered) some pretty good experiences (DNS Made Easy) and some great experiences (CloudFlare).

On L7 stuff, there's a few things no-one tells you about... like if you have your application back onto AWS S3 and serve static files, that the attack can be on your purse as the bandwidth costs can really add up.

It's definitely worth thinking of how to push all costs outside of your little realm. A Varnish cache or Nginx reverse proxy with file system cache can make all the difference by saving your bandwidth costs and app servers.

I personally put CloudFlare in front of my service, but even then I use Varnish as a reverse proxy cache within my little setup to ensure that the application underneath it is really well cached. I only have about 90GB of static files in S3, and about 60GB of that is in my Varnish cache, which means when some of the more interesting attacks are based on resource exhaustion (and the resource is my pocket), they fail because they're probably just filling caches and not actually hurting.

The places you should be ready to add captchas as they really are uncacheable:

* Login pages

* Shopping Cart Checkout pages

* Search result pages

Ah, there's so much one can do, but generally... designing to be highly cacheable and then using a provider who routinely handles big attacks is the way to go.

Re: Ask HN: How do you handle DDoS attacks?

#39
There are many services for HTTP protection, but when you have a custom protocol for a RT service like a game, you are kind of screwed. It's even worst if your game is UDP based.

I used to get attacked huge a load of corrupt UDP packets for a few seconds and that used to hang the main server, wich in 1 or 2 minutes disconnected all my players.

Solution: separate your UDP services from your TCP services in separate applications and servers, also use different type of protection services for each.

The attack still hanged the UDP services, so I started thinking about making a plugin for snort to analyse the traffic and only allow legit protocol packets. I haven't done any of this last idea because the attackers stopped since they noticed that no one was being disconnected.

BTW, for TCP and HTTP I just used any tiny service that protects me from SYN Flood, like Voxility resellers.

Re: Ask HN: How do you handle DDoS attacks?

#40
We need to divide DDoS here in two categories:

-DDoS you can handle (small ones). That anything up to 1 or 2Gbps or 1m packets per second.

-DDoS you can not handle. Anything higher than that.

For the smaller DDoS attacks, you can handle it by adding more servers and using a load balancer (eg. ELB) in front of your site. Both Linode and DigitalOcean will null route your IP address if the attack goes above 100-200Mbps, which is very annoying. Amazon and Google will let you handle on your own (and charge you for it), but you will need quite a few instances to keep up with it.

For anything bigger than that, you have to use a DDoS mitigation service. Even bigger companies do not have 30-40Gbps+ capacity extra hanging around just in case.

I have used and engaged with multiple DDoS mitigation companies and the ones that are affordable and good enough for HTTP (or HTTPS) protection are CloudFlare, Sucuri.net and Incapsula.

-CloudFlare: Is the most popular one and works well for everything but l7 attacks (in my experience). You need to get their paid plan, since the free one does not include ddos protection - they will ask you to upgrade if that happens.

-Sucuri.net: Not as well known as CloudFlare, but they have a very solid mitigation. Have been using them more lately as they are cheaper overall than CloudFlare and have amazing support.

-Incapsula: I used to love them, but their support has been really bad lately. They are on a roll trying to get everyone to upgrade their plans, so that's been annoying. If you can do stuff on your own, they work well.

That's been longer than what I anticipated, but hope it helps you decide.

thanks,

Post reply on HN