Live data from Hacker News

Backpressure explained – the resisted flow of data through software (2019)

medium.com

1–10 of 47 posts

Re: Backpressure explained – the resisted flow of data through software (2019)

#2
There is a subtler issue. Even if your average input and output rates are OK, the lumpiness (stochastic variations) in input and processing rates can cause queues to build up.

In the simplest example, with "well behaved" arrival and processing rates, and a single server (an M/M/1 queue), the average queue length is 1/(1-mu) where the utilization mu = avg arrival rate / avg processing rate. So as the arrival rate approaches the processing rate the avg queue length becomes infinite. In reality, you want to keep the average utilization below 80% to keep queue lengths reasonable.

Re: Backpressure explained – the resisted flow of data through software (2019)

#3
post #2

There is a subtler issue. Even if your average input and output rates are OK, the lumpiness (stochastic variations) in input and processing rates can cause queues to build up. In the simplest example, with "well behaved" arrival and processing rates, and a single server (an M/M/1 queue), the average queue length is 1/(1-mu) where the utilization mu = avg arrival rate / avg processing rate. So as the arrival rate appr…

Use a LIFO with a timeout. When you find a request that exceeds the timeout clear the lifo.

Re: Backpressure explained – the resisted flow of data through software (2019)

#4
Never heard it outside of the context of flow control in networking.

Not a thing in software outside of pieces communicating over a network, not using a protocol which has implicit flow control like TCP.

E.g. words like "the lexer was producing tokens too fast, so the parser applied backpressure" have never been heard.

Re: Backpressure explained – the resisted flow of data through software (2019)

#5
Isn't the ideal solution to make the throttled system faster? Like autoscale horizontally and/or vertically, sharding or just writing better code? Everything in this article is about to cope with back pressure but solving is frequently possible.

Re: Backpressure explained – the resisted flow of data through software (2019)

#6

Never heard it outside of the context of flow control in networking. Not a thing in software outside of pieces communicating over a network, not using a protocol which has implicit flow control like TCP. E.g. words like "the lexer was producing tokens too fast, so the parser applied backpressure" have never been heard.

You won't/can't have it in a direct function call invocation style of programming. E.g. if you have a control loop like "call A, pass result to B, pass result to C" then it's impossible for A to be "too fast."

Network calls are the biggest source of asynchronously queued execution, but you can find models where you have it on a local machine too with multiprocessing. A trivial silly non-network single-machine example might be something like unpacking compressed files than doing [thing] with their contents - maybe you have enough CPU to do them in parallel, but you don't want to blow up your disk by unpacking all of them with no throttling. Even in your lexer/parser example if you wanted to parallelize those steps with a queue in between them, in theory you could have such a huge input that you ran out of memory... in practice, nah, that's not very likely the way you'd do it, or a problem you'd have.

Sometimes "just drop things" or "just make the slow part faster" still aren't really easy/feasible/acceptable even without distributed systems.

I dunno if I'd really call it something like this like the linked article, though "But other forms of backpressure can happen too: for example, if your software has to wait for the user to take some action."

Re: Backpressure explained – the resisted flow of data through software (2019)

#8
In case anyone is interested, I wrote an async/await stream library for JavaScript/Node.js which supports backpressure management. It's heavily tested and used as part of SocketCluster (pub/sub SDK) https://socketcluster.io/ which is also heavily tested.

Re: Backpressure explained – the resisted flow of data through software (2019)

#9

Never heard it outside of the context of flow control in networking. Not a thing in software outside of pieces communicating over a network, not using a protocol which has implicit flow control like TCP. E.g. words like "the lexer was producing tokens too fast, so the parser applied backpressure" have never been heard.

So since you’ve never heard of it, it’s not a thing?

Check out reactive streaming tech like Akka, they’ve been talking about this for well over a decade now, using exactly this language.

Re: Backpressure explained – the resisted flow of data through software (2019)

#10
post #5

Isn't the ideal solution to make the throttled system faster? Like autoscale horizontally and/or vertically, sharding or just writing better code? Everything in this article is about to cope with back pressure but solving is frequently possible.

I'll bet your backpressure mechanism can react at least an order of magnitude faster than your scaling mechanism.
Post reply on HN