Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do you handle DDoS attacks?

news.ycombinator.com

Post reply on HN
Print view View original

81–90 of 114 posts

Re: Ask HN: How do you handle DDoS attacks?

#81

Post by tptacek »

DyslexicAtheist wrote: post #34
tumdum_ wrote: post #2

https://www.cloudflare.com

You mean the biggest MiTM on the web?[0] The only reason why they're not constantly called out by serious infosec folk for their scam is because they hire guys also involved in DefCon/BlackHat planning (try to sneak a hostile talk against Cloudflare past REDACTED[2] who btw is also advising Mr. Robot). It's lobbying at its finest. [0] https://scotthelme.co.uk/tls-conundrum-and-leaving-cloudflar... [1] https://blog.to…

As a:

* Longtime repeat speaker at Black Hat

* Repeat review board member (including this year's), and

* Extreme skeptic of Cloudflare's

I do not believe this is true. If you have a talk that is on topic for Black Hat and is harmful to Cloudflare, you'll get accepted. There's no one person who screens Black Hat talks; it's a panel of people, with several of the longstanding members of that panel (I'm not one of those) being more or less unimpeachable (Mark Dowd, Chris Eagle, Alex Sotirov, Dino Dai Zovi). None of these people are in the tank for Cloudflare. In fact: for most of the review board, none of them give a shit about Cloudflare.

The process isn't perfectly transparent! But it's such that if you submitted a talk, and it got shitcanned before reviewers even saw it, and you made a stink about it on Twitter, people would notice.

I generally agree with your assessment of Cloudflare as a threat to the Internet, for what it's worth. I just don't think you're right that they've gamed Black Hat.

Re: Ask HN: How do you handle DDoS attacks?

#82

Post by [deleted] »

ryanlol wrote: post #79
tumdum_ wrote: post #61

Earlier quoted context omitted.

Yes, I'm well aware that cloudflare is mitm, yet for my needs I've decided that this is not a problem. I can see that you are not happy with what they provide. Luckily theirs service is not forced on you. Neither do you have to use it, nor visit server that use it.

>Yes, I'm well aware that cloudflare is mitm, yet for my needs I've decided that this is not a problem. You're not seeing the full picture here. Using cloudflare is the internet equivalent of dumping hazardous waste, it's not a problem for you but it's a problem for everyone . >I can see that you are not happy with what they provide. Luckily theirs service is not forced on you. Neither do you have to use it, nor visi…

[deleted]

Re: Ask HN: How do you handle DDoS attacks?

#83

Post by cpncrunch »

roblabla wrote: post #30
dineshp2 wrote: post #12

Earlier quoted context omitted.

Tangential, but how do you find OVH? Their hardware, bandwidth, uptime, customer service? I ask because of the conflicting reviews of OVH that a quick google search reveals.

I found OVH's offer to be very good on every point, except customer service. I'm mostly using Kimsufi dedicated servers, and let's just say that when shit goes wrong, you're left alone in the dark. Anecdote: I had my dedicated server suddenly go down because it overheated. Wouldn't come back to life. Two days after submitting a ticket and getting no input, the machine suddenly came back up without any explanation. A…

Kimsufi is a completely different service to OVH's, even though they are owned by the same company, so you can't really compare the two. Kimsufi is dirt cheap and has a reputation for terrible support.

Re: Ask HN: How do you handle DDoS attacks?

#84

Post by DivineTraube »

ryanlol wrote: post #68
DivineTraube wrote: post #67

We (Baqend) use an approach that is somewhat different from what has been proposed here so far: - Every one of our servers rate limits critical resources, i.e. the ones that cannot be cached. The servers autoscale when neccessary. - As rate limiting is expensive (you have to remember every IP/resource pair across all servers) we keep that state in a locally approximated representation using a ring buffer of Bloom fil…

Have you guys been DDoSd before? All that sounds very nice until someone UDP floods you and you get nullrouted. Looks like you're hosting at least some stuff at Hetzner, they're not going to do any filtering for you.

We have not seen any serious DDoS attacks, apart from the ones we created through our own heavy load testing. UDP attacks are not really a problem, since the servers are only accessible by Fastly over TCP using a combination of SSL authentication and IP whitelisting.

Re: Ask HN: How do you handle DDoS attacks?

#85

Post by modoc »

badminton1 wrote: post #76
benchaney wrote: post #57

Earlier quoted context omitted.

Those are problems the CDN solves for you.

CDN is only for static content and I was assumed all your static content would be already on a CDN... it's a standard practice.

CDN is NOT only for static content. Minimally Cacheable, or catchable based on cookie value, content can be cached on a CDN. Also running ALL traffic through a CDN (like CloudFlare or Akamai) allows you to do traffic optimization, FEO, DDOS protection, and much more.

Re: Ask HN: How do you handle DDoS attacks?

#86

Post by justinclift »

Urgo wrote: post #29
tombrossman wrote: post #26

Earlier quoted context omitted.

This contradicts what they claim on their DDOS explanation here: https://www.ovh.co.uk/anti-ddos/mitigation.xml First question to ask them is why was their response different from what practice they advertise? That isn't cool. I would open a new support ticket with them and ask for an explanation since that happened to you recently. If you need something low-cost and dedicated try their SoYouStart range, which is jus…

Here is the result of the support ticket I had with them explaining that the traffic was not an attack and is expected to spike from time to time. Edited only to remove their upsell links. "After looking into the matter, it would seem that our VPS do not have a profile that would accomodate your traffic spikes. However should you switch to a dedicated server, we would be able to apply a a custom profile to your serve…

Sounds like their "Anti-DDOS protection" is bullshit for their VPS plans then, unlike the marketing statements all over the OVH website saying otherwise. :(

Re: Ask HN: How do you handle DDoS attacks?

#88

Post by thatrascaltiger »

* Don't buy service from a single CDN. It's a recipe for disaster because even Akamai have outages, and having a traffic management setup that lets you move partial/entire traffic to a different CDN will let you not only mitigate their outages, but also move traffic to a cheaper/better provider.

* If you can't CDN all your traffic, a CNAME with low TTL that can quickly switch to a CDN/WAF endpoint can be helpful.

* AWS, Azure and GCP all have mitigations for L3 attacks built into their infrastructure. Because you don't know how they operate, or when, don't rely on them. Accept they may break your service and be prepared to have downtime or the means to shift your product quickly if an attack is big enough or presses enough secret buttons.

* Identify and remove all potential means of amplification both at networking/infra and application. This means not exposing your own nameservers or NTP servers publicly, for L7 this is more complicated as it'll depend on how your APIs and products interact with themselves and each other.

* Load test your products often to know what breaking point is and when performance regressions arise with a given amount of resources allocated. Fixing these early may mean you can ride out a DDoS without needing to do anything if it's small enough and your application efficient enough.

Re: Ask HN: How do you handle DDoS attacks?

#89

Post by samplonius »

jaypaulynice wrote: post #69
buro9 wrote: post #38

I've faced DoS attacks for years as I run internet forums. The simple advice for layer 7 (application) attacks: 1. Design your web app to be incredibly cacheable 2. Use your CDN to cache everything 3. When under attack seek to identify the site (if you host more than one) and page that is being attacked. Force cache it via your CDN of choice. 4. If you cannot cache the page then move it. 5. If you cannot cache or mov…

Have you heard about cache busting? Someone just needs to request a page that's not cached and the request will always hit your web servers.

If there are 90GB of static files, and 60GB are in the Varnish cache, cache busting will be pretty ineffective.

Re: Ask HN: How do you handle DDoS attacks?

#90

Post by Kenji »

buro9 wrote: post #38

I've faced DoS attacks for years as I run internet forums. The simple advice for layer 7 (application) attacks: 1. Design your web app to be incredibly cacheable 2. Use your CDN to cache everything 3. When under attack seek to identify the site (if you host more than one) and page that is being attacked. Force cache it via your CDN of choice. 4. If you cannot cache the page then move it. 5. If you cannot cache or mov…

Uh, stupid question but how do you cache a website like for example this comment thread on hackernews? Suppose a DDoSer calls this comment thread a lot of times. The request has to go through to the server because when I hit F5 or post a comment myself, I see the comments in realtime. How do you handle that exactly? Does caching for a few seconds help already, or does the backbone push updated sites to the CDN server? I have no experience in DDos mitigation.
Post reply on HN