Update from Elon: https://twitter.com/elonmusk/status/1675214274627530754 > Rate limits increasing soon to 8000 for verified, 800 for unverified & 400 for new unverified lolwut. Quoting a post from Mastodon: https://mstdn.social/@maxkennerly/110640373859329500 > LOL this is not how you deal with "data scraping," this is how you deal with a catastrophic loss of system capacity. > You limit data scraping by blocking th…
This isn't accurate at all.
Standard operating procedure for large-scale scraping is to use a botnet, it's a sybil attack where you can have tens or even hundreds of thousands of IPs ticking away at 10 requests per minute or some such. Done correctly, it's nearly impossible to detect which nodes are complicit from their access patterns or user agents.
That said, rate limits work relatively poorly against this. A proof of work approach like HashCash might work though.