Queues don't fix overload (2014)
1–10 of 156 posts
Re: Queues don't fix overload (2014)
#2(2014)
Fwiw, sinks have overflows: https://www.thespruce.com/the-purpose-of-a-sink-overflow-271...
The computer equivalent is probably load shedding
Re: Queues don't fix overload (2014)
#3(2014) Fwiw, sinks have overflows: https://www.thespruce.com/the-purpose-of-a-sink-overflow-271... The computer equivalent is probably load shedding
The sink overflow goes to the same drain, so it doesn't really work in this analogy! (Or it works very well, depending on your perspective).
Re: Queues don't fix overload (2014)
#4(2014) Fwiw, sinks have overflows: https://www.thespruce.com/the-purpose-of-a-sink-overflow-271... The computer equivalent is probably load shedding
The sink overflow goes to the same drain, so it doesn't really work in this analogy! (Or it works very well, depending on your perspective).
The overflow goes to the same drain, but further down, past a bottleneck that's designed to be there for a logical reason. It's a good example of real world flow control considerations vs paper theory.
Re: Queues don't fix overload (2014)
#5Depends.
An overflowing queue that drops jobs from the front while backend chugs along as fast as it can, is for many cases a better outcome than the backend overloading and everything grinding to a halt.
Compare it to a physical service desk. The one that has a queue, serves one person at a time, and people arriving will try again another day if the queue is too long. The one without queue has people fistfighting over who gets to go first, and no one ends up getting service.
Re: Queues don't fix overload (2014)
#6So far as I know there is no theoretical alternative to load shedding or increasing handling capacity if your average request arrival rate is greater than your average request handling rate. At least, not if you want to handle every accepted request using a finite queue[1]. It would appear that with an unbounded queue every request will eventually be handled, but with an unbounded latency guarantee. Which appears equivalent to “grinds to a halt” for sufficient n.
However, that may very well change with fair queueing. If unbounded queue storage is available and you can prioritize requests in a timely fashion, then instead of shedding excess requests they can go into unbounded queue limbo instead while still meeting the SLA for priority requests. I imagine there is developed queueing theory for that.
[1] I have been taught here though that classical queuing theory isn’t adequate for all cases. I think it is here, but I will gratefully accept correction if I’m wrong.
Re: Queues don't fix overload (2014)
#7The other thing to bear in mind about queues is that once they start showing of symptoms of something being wrong, collapse might be just around the corner or it might not be depending on the nature of the load.
When congestion spikes start showing it is helpful to know some queuing theory to estimate how close the situation is to eating someone's weekend. Congestion collapses are an interesting time because most people don't know queue theory or how to reason using balance equations, it is possible to misdiagnose the problem or waste a stressful few days trying to work out a congestion situation by experiment.
Re: Queues don't fix overload (2014)
#8This is a weird article because it points out that queues don’t solve overload but neither do load shedding or back pressure.
All 3 techniques are just different trade offs on what to do in the face of overload. All 3 have negative ramifications for the users of the system. Load shedding reduces availability, back pressure increases complexity and queues increase latency.
In “critical” systems you need all 3. And all 3 can be overloaded. Frankly, your load shedding or back pressure system is probably implemented on a queue one layer down the abstraction.
Re: Queues don't fix overload (2014)
#9Depends. An overflowing queue that drops jobs from the front while backend chugs along as fast as it can, is for many cases a better outcome than the backend overloading and everything grinding to a halt. Compare it to a physical service desk. The one that has a queue, serves one person at a time, and people arriving will try again another day if the queue is too long. The one without queue has people fistfighting ov…
That's loadshedding.
Re: Queues don't fix overload (2014)
#10Another thing. Queues often lack prioritization of messages; for instance, the importance of a new user signup may be overlooked compared to an address update.