If interested, I wrote a different type of rate limiter in my spare time for my use case.
Tollbooth: HTTP rate limiter middleware in Go
11–20 of 21 posts
Re: Tollbooth: HTTP rate limiter middleware in Go
#12I was writing an netfilter/iptables rate limiter and a nginx rate limiter (both able to handle quite a lot of requests) but all the system admins I talked to shamed me for trying to rate limit. They raised all kinds of points about blocking access to large companies or small countries and it not being effective for DDoS anyway. Any thoughts on this?
Re: Tollbooth: HTTP rate limiter middleware in Go
#13It looks to me like the map of rate buckets will grow unbounded and never be cleaned up?
Re: Tollbooth: HTTP rate limiter middleware in Go
#14If interested, I wrote a different type of rate limiter in my spare time for my use case. http://badactor.org
I really like your simple splash demo. Good work, I might even check out your code.
Re: Tollbooth: HTTP rate limiter middleware in Go
#15I was writing an netfilter/iptables rate limiter and a nginx rate limiter (both able to handle quite a lot of requests) but all the system admins I talked to shamed me for trying to rate limit. They raised all kinds of points about blocking access to large companies or small countries and it not being effective for DDoS anyway. Any thoughts on this?
The only effective counter against a proper DDoS is to bring more bandwidth than your attacker has.
A small site is unlikely to attract a proper DDoS, but will get hit by the usual misbehaving web scrapers, broken clients in a silly loop (I'm looking at you, iTunes) and keyboards with super sensitive F5 key. In those cases, serving an occasional 503 might work better than the alternatives.
Re: Tollbooth: HTTP rate limiter middleware in Go
#16 Nah, go's shits smells around HN.
Re: Tollbooth: HTTP rate limiter middleware in Go
#17I was writing an netfilter/iptables rate limiter and a nginx rate limiter (both able to handle quite a lot of requests) but all the system admins I talked to shamed me for trying to rate limit. They raised all kinds of points about blocking access to large companies or small countries and it not being effective for DDoS anyway. Any thoughts on this?
A bandwidth rate limiter is indeed of dubious utility. There's a lot of ways that can go wrong, and there's a lot of ways it can fail to do what the user expected anyhow, and there's a lot of ways it can do both.
But there's a lot of things that can be sensibly rate limited, such as logins attempts to a given account. Now that has its own considerations, too, if pushed to the limit... you'd prefer that an attacker can't lock down your service just by spuriously trying to log in to all your accounts 5 times every 5 minutes or something. But loud downtime (which you can then react to) may be preferable to getting your users silently hacked.
I definitely agree that they are less useful than they look at first glance, a great deal more complicated than you'd like, and more subtle than you'd think. But they can still be a useful tool.
Re: Tollbooth: HTTP rate limiter middleware in Go
#18Serious question: why would you not use HAProxy, nginx or similar software which arguably are more battle tested in this domain?
The best reason I can imagine is so that it can all be wrapped in your portable Go binary. Because of the portability aspect, there are a lot of use cases for Go similar to Java where there is an incentive to do everything "in Go".
Re: Tollbooth: HTTP rate limiter middleware in Go
#19It looks to me like the map of rate buckets will grow unbounded and never be cleaned up?
Looks to be the case based on the config. I'm suprised storage wasn't abstracted to accommodate redis or the like which is essential in a load balanced scenario.
The API doesn't seem quite idiomatic either, I'd expect to create a struct containing options and a function that closes over the http.Handler interface e.g. func(l *Limiter) Limit(next http.Handler) http.Handler or a function that takes options and a next http.Handler that creates a struct implementing http.Handler.
Re: Tollbooth: HTTP rate limiter middleware in Go
#20If interested, I wrote a different type of rate limiter in my spare time for my use case. http://badactor.org
I really like your simple splash demo. Good work, I might even check out your code.
The code is solid. I gauge this by the simple fact that when I review the project, I don't absolutely hate it ;-)
I need to circle back around as there are some features I'd like to add. PRs are welcome.