Backpressure explained – the resisted flow of data through software (2019)
1–10 of 47 posts
Re: Backpressure explained – the resisted flow of data through software (2019)
#2In the simplest example, with "well behaved" arrival and processing rates, and a single server (an M/M/1 queue), the average queue length is 1/(1-mu) where the utilization mu = avg arrival rate / avg processing rate. So as the arrival rate approaches the processing rate the avg queue length becomes infinite. In reality, you want to keep the average utilization below 80% to keep queue lengths reasonable.
Re: Backpressure explained – the resisted flow of data through software (2019)
#3There is a subtler issue. Even if your average input and output rates are OK, the lumpiness (stochastic variations) in input and processing rates can cause queues to build up. In the simplest example, with "well behaved" arrival and processing rates, and a single server (an M/M/1 queue), the average queue length is 1/(1-mu) where the utilization mu = avg arrival rate / avg processing rate. So as the arrival rate appr…
Re: Backpressure explained – the resisted flow of data through software (2019)
#4Not a thing in software outside of pieces communicating over a network, not using a protocol which has implicit flow control like TCP.
E.g. words like "the lexer was producing tokens too fast, so the parser applied backpressure" have never been heard.
Re: Backpressure explained – the resisted flow of data through software (2019)
#5Re: Backpressure explained – the resisted flow of data through software (2019)
#6Never heard it outside of the context of flow control in networking. Not a thing in software outside of pieces communicating over a network, not using a protocol which has implicit flow control like TCP. E.g. words like "the lexer was producing tokens too fast, so the parser applied backpressure" have never been heard.
Network calls are the biggest source of asynchronously queued execution, but you can find models where you have it on a local machine too with multiprocessing. A trivial silly non-network single-machine example might be something like unpacking compressed files than doing [thing] with their contents - maybe you have enough CPU to do them in parallel, but you don't want to blow up your disk by unpacking all of them with no throttling. Even in your lexer/parser example if you wanted to parallelize those steps with a queue in between them, in theory you could have such a huge input that you ran out of memory... in practice, nah, that's not very likely the way you'd do it, or a problem you'd have.
Sometimes "just drop things" or "just make the slow part faster" still aren't really easy/feasible/acceptable even without distributed systems.
I dunno if I'd really call it something like this like the linked article, though "But other forms of backpressure can happen too: for example, if your software has to wait for the user to take some action."
Re: Backpressure explained – the resisted flow of data through software (2019)
#7Re: Backpressure explained – the resisted flow of data through software (2019)
#8Re: Backpressure explained – the resisted flow of data through software (2019)
#9Never heard it outside of the context of flow control in networking. Not a thing in software outside of pieces communicating over a network, not using a protocol which has implicit flow control like TCP. E.g. words like "the lexer was producing tokens too fast, so the parser applied backpressure" have never been heard.
Check out reactive streaming tech like Akka, they’ve been talking about this for well over a decade now, using exactly this language.
Re: Backpressure explained – the resisted flow of data through software (2019)
#10Isn't the ideal solution to make the throttled system faster? Like autoscale horizontally and/or vertically, sharding or just writing better code? Everything in this article is about to cope with back pressure but solving is frequently possible.