Writing Well-Documented Code – Learn from Examples

I've found that hence e.g. for rather complex statements where the code isn't lacking information, not commenting is the correct approach.

Whereas sometimes, when I'm forced to write exceptional code because of e.g. a workaround, a comment can nicely introduce my future self and other readers to the context.

Secondly, the implementation for the token bucket seems to just be bad. A simpler implementation is as follows:

- Input number of tokens and refill time.

- Output the constant state describing the limiter: the max number of tokens and the time to get a single token (= refill time / size, ignoring rounding issues which shouldn’t generally matter if your times are in nanos)

- The only mutable state is a single time instant: the time when the limiter becomes full

- to attempt to take a token, compute max(0,full_time - now) / new_token_time and if this is less than or equal to the size of the bucket minus the number of tokens you want, you may withdraw and set full_time := max(full_time, now) + tokens_to_withdraw * new_token_time

If that division is concerning then you should either attempt to withdraw from the rate limiter less frequently or you should not be using one at all.

The rest of the code is about scheduling events to happen when there are free tokens in the ratelimiter. This is a different problem and also over-complicated. You should either poll and retry periodically with your most important event (rather than whatever event you thought was most important when the bucket became empty) though you then want to be careful about stale tasks never running, or you should just have a queue of pairs (Task, cost) where cost is an int. after withdrawing tokens from the ratelimiter or when the queue transitions from empty to non-empty, peek the top item and set a timer for rl.full_time - rl.new_token_time * (rl.size - cost) and when the timer fires (or immediately if the time was in the past) you can dequeue, withdraw cost tokens, and run that top task (then repeat.)

I've changed BucketReduction to be a pair, and fixed the strange behavior on error. Swapping "fromBurst" and "auto_replenish" would reduce the number of throttling events. This solution does have more branches than the original, hence readability vs performance. It's still not exception safe, auto_replenish could throw and leave the state undefined.

    BucketReduction reduce(long tokens) {
        long fromBurst = min(tokens, burst);
        burst -= fromBurst;
        tokens -= fromBurst;

        if (tokens > budget) {
            auto_replenish();
        }

        long fromBudget = min(tokens, budget);
        budget -= fromBudget;
        tokens -= fromBudget;

        long totalConsumed = fromBurst + fromBudget;
        if (tokens == 0) {
            return new BucketReduction(Success, totalConsumed);
        }

        if (totalConsumed + tokens > size) {
            return new BucketReduction(OverConsumption, totalConsumed);
        }

        return new BucketReduction(Failure, totalConsumed);
    }