I coded up something like Fair a couple 1-2 years ago: https://github.com/JensRantil/conc It's alpha software, but maybe interesting to someone don't know.
Fair: A Go library for serving resources fairly
41–44 of 44 posts
Re: Fair: A Go library for serving resources fairly
#42Shouldn’t this be built into a queue somehow? I’d love to see a queuing solution like SQS but has a built in fairness, where you can fully utilize a capacity but as soon as, let’s say customers compete on resources, some fairness kicks in. Is there anything like that?
Multiple queues?
Re: Fair: A Go library for serving resources fairly
#43Does anyone have some real-world use cases for something like this? The algorithm is cool but I'm struggling to see where this is applicable.
The text suggests a method for managing GPU or rate-limited resources across multiple clients. It highlights the problem of spikey workloads, where a client might generate a large number of events (e.g., from a CSV upload) causing resource starvation. The text advises against using naive solutions like FIFO, which could disadvantage clients with steady live traffic.
Re: Fair: A Go library for serving resources fairly
#44> Since the state is stored in a multi-level Bloom Filter style data structure, the memory needed is constant and does not scale with the number of clients. Constant memory, but those hashes will take up CPU cycles. If you're running a workload that completes sub 20 milliseconds, these cycles spent hashing may not be worth it over, say, a constant-time admission control like token bucket.
The world is full of trade-offs. I don't think the author intended to solve CPU-bound workloads like that. Or have they claimed that?
Not just CPU-bound, but IO-bound workloads can also complete within milliseconds.
> have they claimed that?
The mention of "token bucket" in the project readme is why I wrote the comment I did.
... FAIR [only throttles] when there's a genuine shortage of resources as opposed to the approaches like token bucket or leaky bucket which may reject requests ...