Live data from Hacker News

Someone attacked our company

usefathom.com

21–30 of 112 posts

Re: Someone attacked our company

#21
PHP Laravel on Amazon Lambda to count pageviews? Are you sure it's not just a regular customer and not a DDOS?

> privacy-first analytics solution [...] The only downside of this is that we need to keep access logs (IP & User-Agent, no browsing history) for 24 hours

Keeping IPs don't make you super privacy friendly.

Re: Someone attacked our company

#24

PHP Laravel on Amazon Lambda to count pageviews? Are you sure it's not just a regular customer and not a DDOS? > privacy-first analytics solution [...] The only downside of this is that we need to keep access logs (IP & User-Agent, no browsing history) for 24 hours Keeping IPs don't make you super privacy friendly.

Awesome, I'd love some advice if you're willing. So here's our situation.

* We're getting hit with a huge DDoS attack, repeatedly over 3 weeks, with no sign of stopping

* With zero access logs, there was no way to find patterns in the attack, and we had no way to block it

* Our service was going offline during these attacks

* We introduced access logs that are auto-deleted after 24 hours. We redacted all information about the site/page/activity etc. but keep IP & User Agent for pattern matching

* We were then able to identify a pattern and block the attack on Saturday

* Without access logs (even redacted ones), this wasn't possible

I was hoping a more senior engineer on Hacker News would comment and I can't wait to hear how you'd do it. I have no experience in DDoS protection at all, and this seems like the only possible way. Even rate limiting requires storing IP addresses. But if you know a more privacy-focused way to block these attacks, I'm sure I'll buy you a few beers when we hang out.

Re: Someone attacked our company

#26

PHP Laravel on Amazon Lambda to count pageviews? Are you sure it's not just a regular customer and not a DDOS? > privacy-first analytics solution [...] The only downside of this is that we need to keep access logs (IP & User-Agent, no browsing history) for 24 hours Keeping IPs don't make you super privacy friendly.

Awesome, I'd love some advice if you're willing. So here's our situation. * We're getting hit with a huge DDoS attack, repeatedly over 3 weeks, with no sign of stopping * With zero access logs, there was no way to find patterns in the attack, and we had no way to block it * Our service was going offline during these attacks * We introduced access logs that are auto-deleted after 24 hours. We redacted all information…

I would probably start by looking for third-parties to help manage to problems, at least unless they all end up at the application layer. Throw it beyond Cloudflare and talk to them, and if they're not the right fit, try someone else.

Re: Someone attacked our company

#27
post #26

Earlier quoted context omitted.

Awesome, I'd love some advice if you're willing. So here's our situation. * We're getting hit with a huge DDoS attack, repeatedly over 3 weeks, with no sign of stopping * With zero access logs, there was no way to find patterns in the attack, and we had no way to block it * Our service was going offline during these attacks * We introduced access logs that are auto-deleted after 24 hours. We redacted all information…

I would probably start by looking for third-parties to help manage to problems, at least unless they all end up at the application layer. Throw it beyond Cloudflare and talk to them, and if they're not the right fit, try someone else.

There is nothing wrong with this approach but exactly this mentality lead to the actual centralized internet where only a few peers handle most of the traffic.

Re: Someone attacked our company

#28

PHP Laravel on Amazon Lambda to count pageviews? Are you sure it's not just a regular customer and not a DDOS? > privacy-first analytics solution [...] The only downside of this is that we need to keep access logs (IP & User-Agent, no browsing history) for 24 hours Keeping IPs don't make you super privacy friendly.

Awesome, I'd love some advice if you're willing. So here's our situation. * We're getting hit with a huge DDoS attack, repeatedly over 3 weeks, with no sign of stopping * With zero access logs, there was no way to find patterns in the attack, and we had no way to block it * Our service was going offline during these attacks * We introduced access logs that are auto-deleted after 24 hours. We redacted all information…

> * We're getting hit with a huge DDoS attack, repeatedly over 3 weeks, with no sign of stopping

I've read the full blog post, I am not convinced it's a DDoS attack. Traffic patterns for web analytics will come from over the place and will look like a DDoS when it's not. For example, a customer misplacing their analytics in a JS loop and having a moderate traffic blog will generate billions of requests from all over the globe. Event tracking can be billions of requests as well by themselves. Never attribute to malice that which is adequately explained by simpler means.

> * With zero access logs, there was no way to find patterns in the attack, and we had no way to block it

It's a loss of time and energy. Your system should be able to handle these billions of requests.

> * We were then able to identify a pattern and block the attack on Saturday

Was it specific accounts?

> * Without access logs (even redacted ones), this wasn't possible

You can one-way hash the IP. So you can still look for pattern but you've lost the actual IP. And same one-way hash IP can block whichever IP seems devious in your firewall. (like md5 can be enough.)

> But if you know a more privacy-focused way to block these attacks, I'm sure I'll buy you a few beers when we hang out.

Haha no need to. But come say hi if you ever in Austin. julien _at_ serpapi.com.

Re: Someone attacked our company

#30
post #3

What would the motivation be for such an attack? Any thoughts on who could be behind it?

And why? What is "Fathom", what do they do that could have triggered such an attack? Or was that an attack to deliberately force them to turn to Cloudflare for "help"?
Post reply on HN