Live data from Hacker News

Backpressure explained – the resisted flow of data through software (2019)

medium.com

11–20 of 47 posts

Re: Backpressure explained – the resisted flow of data through software (2019)

#11
post #5

Isn't the ideal solution to make the throttled system faster? Like autoscale horizontally and/or vertically, sharding or just writing better code? Everything in this article is about to cope with back pressure but solving is frequently possible.

In practice, yeah, this is the fix in most systems because in a microservice context it would feel gross to have a slow consumer reach out and throttle a fast producer.

It’s still important to understand though because autoscaling has its own back pressure that you run into eventually. Accounts may have quotas or regions might be out of a given instance type, or out of a type at your preference spot price.

Re: Backpressure explained – the resisted flow of data through software (2019)

#12

Never heard it outside of the context of flow control in networking. Not a thing in software outside of pieces communicating over a network, not using a protocol which has implicit flow control like TCP. E.g. words like "the lexer was producing tokens too fast, so the parser applied backpressure" have never been heard.

You won't/can't have it in a direct function call invocation style of programming. E.g. if you have a control loop like "call A, pass result to B, pass result to C" then it's impossible for A to be "too fast." Network calls are the biggest source of asynchronously queued execution, but you can find models where you have it on a local machine too with multiprocessing. A trivial silly non-network single-machine example…

Ehhh I think labeling user input as backpressure, because the software is waiting for _input_, is somewhere between confusing and inaccurate. When I have seen backpressure discussed in my day job, it has always involved a (theoretical or real) slow consumer, and therefore some queue in front of that consumer. I agree that "networking" or not, is irrelevant.

Re: Backpressure explained – the resisted flow of data through software (2019)

#13
post #5

Isn't the ideal solution to make the throttled system faster? Like autoscale horizontally and/or vertically, sharding or just writing better code? Everything in this article is about to cope with back pressure but solving is frequently possible.

Ideally yes, in practice… no.

In reality you hit amdahl’s law pretty quickly.

Re: Backpressure explained – the resisted flow of data through software (2019)

#14

Never heard it outside of the context of flow control in networking. Not a thing in software outside of pieces communicating over a network, not using a protocol which has implicit flow control like TCP. E.g. words like "the lexer was producing tokens too fast, so the parser applied backpressure" have never been heard.

The general pattern is a fixed set of resources that are consumed/retired at a fixed maximum rate, where the optimal design consistently gets as close to the maximum rate as possible without exceeding the resource limit. There are (at least) two other places in software where backpressure is used, both commonly found in the database world:

Storage I/O scheduling, which unlike the network case is often not interrupt-driven. As you approach the IOPS rate limit for the system due to high priority tasks, the rate of IOPS scheduled for low priority tasks like background write-back in databases is dynamically reduced to maintain headroom for high priority tasks. This is implemented as backpressure on low-priority tasks.

In query processing, a query as a user would understand it can materialize upwards of millions of sub-operations on the same server that can run concurrently given adequate system resources. The number of sub-operations that can be in-flight and retired per second is approximately fixed and shared across all concurrent user queries. For large queries, you incrementally materialize these sub-operations at a rate based on the instantaneous capacity of the system to handle new sub-operations. This is backpressure based on execution slot (and related memory) availability.

Writing software for barrel processors takes this to the extreme. The entire software design principle is to consistently generate fine-grained concurrent threads of execution at runtime that are close to the hardware concurrency limit globally (which is very high) but never exceeds it. Highly optimized code quickly gets pretty weird but it is basically all backpressure mechanics to maintain throughput.

Re: Backpressure explained – the resisted flow of data through software (2019)

#15

Never heard it outside of the context of flow control in networking. Not a thing in software outside of pieces communicating over a network, not using a protocol which has implicit flow control like TCP. E.g. words like "the lexer was producing tokens too fast, so the parser applied backpressure" have never been heard.

"Nobody drives here, there's too much traffic." is a classic backpressure quote.

Re: Backpressure explained – the resisted flow of data through software (2019)

#16
I like the article, but I am not sure that I agree with the terminology: I would not call "buffering" a form of back pressure. Imho there is really only one type of back pressure: the one the author calls "control". The other 2 are just ways to "release" pressure.

Re: Backpressure explained – the resisted flow of data through software (2019)

#17
post #16

I like the article, but I am not sure that I agree with the terminology: I would not call "buffering" a form of back pressure. Imho there is really only one type of back pressure: the one the author calls "control". The other 2 are just ways to "release" pressure.

Edit: I think the article also misses one of the most important ways to release pressure. And that is scaling throughput either horizontally (e.g. by adding more servers) or vertically (e.g. by optimization of software)

Re: Backpressure explained – the resisted flow of data through software (2019)

#18
post #5

Isn't the ideal solution to make the throttled system faster? Like autoscale horizontally and/or vertically, sharding or just writing better code? Everything in this article is about to cope with back pressure but solving is frequently possible.

I'll bet your backpressure mechanism can react at least an order of magnitude faster than your scaling mechanism.

Serverless systems are pretty decent at scaling quickly these days. The problem is rarely lack of servers in my experience, though. You usually run out of database connections or some other bottleneck first.

Re: Backpressure explained – the resisted flow of data through software (2019)

#19
post #5

Isn't the ideal solution to make the throttled system faster? Like autoscale horizontally and/or vertically, sharding or just writing better code? Everything in this article is about to cope with back pressure but solving is frequently possible.

it doesn't matter. as long as there is a rate mismatch over a sufficiently long timescale, queues will build up and consume memory, latency will increase, and depending on how things are structured overall throughput will go down because of scheduling overhead and memory pressure.

this is actually an inherent problem _within_ horizontally scaled services that have cross dependencies.

so no, you should really implement backpressure.

Re: Backpressure explained – the resisted flow of data through software (2019)

#20
WebSockets is an interesting case where the underlying transport (TCP) provides backpressure for free, but the way the API was designed in browsers throws it away. For example, it's trivial to fill your device's memory by opening a large local file in your browser and attempting to stream it to a server in a tight WebSocket send loop.

I'm not sure if there was an alternative when WebSockets was designed. Did we even have promises yet?

This sort of thing is solved nowadays with WhatWG streams. They're a bit verbose to work with but I've been impressed with the design.

Post reply on HN