Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding the recent DDoS attack against Read the Docs

about.readthedocs.com

Post reply on HN
Print view View original

11–20 of 73 posts

Re: Understanding the recent DDoS attack against Read the Docs

#11

Post by davidfischer »

tescreal wrote: post #7
davidfischer wrote: post #4

Earlier quoted context omitted.

I'm the author of the blog. I don't know. Internally, we were half joking that we were going to get ransom notice, but we never did. The only thing that sort of correlates with this attack is that before it started, we began rolling out some slightly more aggressive rate limits one by one. This was mostly because anytime any new "company" thinks they're going to catchup with Claude/OpenAI, they scrape us very aggress…

Good to know. I use your site (with a manual transmission user-agent) often, and it's fantastic. Thanks for your work and the writeup!

I've never seen the phrase "manual transmission user-agent". Using your own browser yourself is the new stick shift. Love it.

Re: Understanding the recent DDoS attack against Read the Docs

#13

Post by Onavo »

A more interesting question is, what exactly do the attackers gain from hitting read the docs? Most of their docs hosting is static/easily CDN cached. Unlike database bound sites, you would need a lot more traffic to overload pure/mostly static hosting. Maybe it's a malicious AI lab looking to deny their competitors training data? As far as infosec profiling goes, this is probably the oddest case I have heard of.

I am thinking it's probably an AI lab that misconfigured their data scraper (made it too agentic) and it ended up looking like a DDoS.

The new generation of scrapers are all agentic and self healing. (As an example see YC's https://parse.bot)

Re: Understanding the recent DDoS attack against Read the Docs

#14

Post by davidfischer »

Onavo wrote: post #13

A more interesting question is, what exactly do the attackers gain from hitting read the docs? Most of their docs hosting is static/easily CDN cached. Unlike database bound sites, you would need a lot more traffic to overload pure/mostly static hosting. Maybe it's a malicious AI lab looking to deny their competitors training data? As far as infosec profiling goes, this is probably the oddest case I have heard of. I a…

Author here. This was not a misconfigured data scraper. We see those every week[1]. This attack wasn't scraping useful content. It was almost entirely 404s and 302s and pulled virtually zero real docs. It specifically looked for URLs not served by the CDN and when it found a pattern, did millions of variations of it. Whether built by an AI or not, it was designed to cause outages and financial damage from autoscaling. However, as others have suggested, we may have been a test run for a real target.

[1] https://about.readthedocs.com/blog/2024/07/ai-crawlers-abuse...

Re: Understanding the recent DDoS attack against Read the Docs

#15

Post by braiamp »

Onavo wrote: post #13

A more interesting question is, what exactly do the attackers gain from hitting read the docs? Most of their docs hosting is static/easily CDN cached. Unlike database bound sites, you would need a lot more traffic to overload pure/mostly static hosting. Maybe it's a malicious AI lab looking to deny their competitors training data? As far as infosec profiling goes, this is probably the oddest case I have heard of. I a…

> Most of their docs hosting is static/easily CDN cached

The article says

> and it purposefully attacked areas that bypassed caching

So that doesn't work. Also, it seems that they were trying to cause financial harm, not to take down the infrastructure but to make it costly for the org itself. That's smart.

Re: Understanding the recent DDoS attack against Read the Docs

#19

Post by gkoberger »

tescreal wrote: post #2

I'm curious if anybody could speculate who would be attacking a documentation silo, and to what end?

I run a similar service, and we get almost daily attacks like this. Sometimes it's a specific high-profile customer, other times it's broader.

I can't speak for RTD, but I think it's less "documentation site" and more just that we sit on the domains of high-profile products and the tools are just looking for any hole they can find?

Often it's even the company themselves, for whatever reason (security research, etc).

Re: Understanding the recent DDoS attack against Read the Docs

#20

Post by Symbiote »

fn-mote wrote: post #18

There’s an assumption that turning on Cloudflare’s “under attack” mode would mitigate the attack. Given how adaptive the rest of the attack was, I would be very curious to find out how it would approach that obstacle.

That has only partially mitigated much smaller attacks (residential proxy scraping etc) on my employer's site.

We're currently on the "Business" plan, but I'm coming to the conclusion that we need to upgrade to the "Enterprise Advantage" plan for the JA3/4 fingerprinting and detection ID features.

I get put off by "Contact Sales" pricing.

Post reply on HN