Live data from Hacker News

Ask HN: How can I tell if a site is being rate limited?

news.ycombinator.com

21–29 of 29 posts

Re: Ask HN: How can I tell if a site is being rate limited?

#21

Just curious -- if a country didn't want a site to be seen, why would they rate-limit it instead of just blocking connections completely? Is that a thing countries do, like if they want to "punish" a site while retaining plausible deniability?

Yes: https://news.ycombinator.com/item?id=22541960

I have seen this actually happen IRL

Re: Ask HN: How can I tell if a site is being rate limited?

#22
The thing about rate limits is, there are often hard limits. Try querying the site a bunch and see what the data transfer rate is. if it's consistently incapable of going above say, 128K/sec, you've got your rate limit.

Other times, they'll allow for bursts but sustained get rate limited - these can be detected just the same depending on their approach; if it's per connection (and not IP) you may have to find a big file to download.

It also helps to establish that traffic to other sites is not rate limited, and to use 3rd party checkers that check for connection rate across the globe. Or DIY and spin up a bunch of EC2 machines that are geographically diverse.

Re: Ask HN: How can I tell if a site is being rate limited?

#23
If the government entity really is throttling that site, assuming they are competent, you're unlikely to find conclusive proof that they're actually throttling. Compare this with a full on blackout (i.e. Great Firewall of China, where that's obvious proof of government intervention); the fact that they're allowing some requests to go thru means that the meddling is likely engineered to not be detectable. Best you can find is maybe the speeds are slower when accessed from within the country, but they can always plausibly deny it.

Re: Ask HN: How can I tell if a site is being rate limited?

#24

The thing about rate limits is, there are often hard limits. Try querying the site a bunch and see what the data transfer rate is. if it's consistently incapable of going above say, 128K/sec, you've got your rate limit. Other times, they'll allow for bursts but sustained get rate limited - these can be detected just the same depending on their approach; if it's per connection (and not IP) you may have to find a big f…

I don't know about this, I've heard that limiting is often intentionally inconsistent because this makes people less likely to visit. I think Harlow did some similar stuff with his monkeys.

Re: Ask HN: How can I tell if a site is being rate limited?

#25

I can definitely notice an improvement in performance when I use a VPN, however it would be cool to see where things are being delayed. Is traceroute or curl something that could help here?

I ran a curl test and noticed that the TCP connect time is where things are getting hung up, taking over 80 seconds! Does anyone know if this is indicative of deliberate rate limiting or just a bad peering connection?

Possibly neither, that is, there are many possible reasons.

for example, stateful firewalls can only track a certain number of connections. If you don't have a public IP, or if they are trying to protect you, it may be necessary to wait for a connection resource to come available. Connections are generally expensive, different from data transfer, for these and other reasons, such as port starvation, syn flood defenses inappropriately targeting you, so many others.

Re: Ask HN: How can I tell if a site is being rate limited?

#26

Have some fun at the same time: 1) get a vpn account (proton, other) 2) hack your worst enemy's wifi with your favourite tools 3) while outside your enemy's home, boot a linux live distro 4) change your computer's name to your enemy's name 5) connect to the enemy's wifi 6) try to visit all the sites banned by your country 7) insult your country's leader in a public forum using your enemy's name 8) test the speed of t…

He should also modify the said enemy's browser cache for this to work well.

Re: Ask HN: How can I tell if a site is being rate limited?

#27

Just curious -- if a country didn't want a site to be seen, why would they rate-limit it instead of just blocking connections completely? Is that a thing countries do, like if they want to "punish" a site while retaining plausible deniability?

Yes: https://news.ycombinator.com/item?id=22541960 I have seen this actually happen IRL

Thanks so much for the link. That was really educational, answered my question completely.

Re: Ask HN: How can I tell if a site is being rate limited?

#28

Have some fun at the same time: 1) get a vpn account (proton, other) 2) hack your worst enemy's wifi with your favourite tools 3) while outside your enemy's home, boot a linux live distro 4) change your computer's name to your enemy's name 5) connect to the enemy's wifi 6) try to visit all the sites banned by your country 7) insult your country's leader in a public forum using your enemy's name 8) test the speed of t…

This is very insightful. Tip of the hat to you!
Post reply on HN