Backpressure explained – the resisted flow of data through software (2019) • Hacker News Board

Re: Backpressure explained – the resisted flow of data through software (2019)

#41

Post by sly010 » Wed, Mar 27, 2024, 1:41 PM UTC

coldtea wrote: post #26

sly010 wrote: post #17

Earlier quoted context omitted.

Edit: I think the article also misses one of the most important ways to release pressure. And that is scaling throughput either horizontally (e.g. by adding more servers) or vertically (e.g. by optimization of software)

The first is not always possible (not all systems are elastic or can be, due to money/resources), and the second is not really a way to handle back-pressure. You could have back-pressure in the most optimized system.

Note that I meant "handling (forward) pressure (or preventing back-pressure) can be done by simply having your system be performant enough. Of course this is not a dynamic property you can adjust at runtime, just wanted to add for the sake of completeness, because it should be part of the mindset. Sometimes a database falling over simply needs a few queries optimized.

> You could have back-pressure in the most optimized system.

Most batch processing systems. But you don't want back pressure in interactive or real-time systems, like graphics, gaming, audio, cars, planes, or even just real time collaboration systems (e.g. Figma). In all these cases back pressure is to be avoided.

Re: Backpressure explained – the resisted flow of data through software (2019)

#42

Post by gpderetta » Wed, Mar 27, 2024, 2:28 PM UTC

geggo98 wrote: post #36

kazinator wrote: post #4

Never heard it outside of the context of flow control in networking. Not a thing in software outside of pieces communicating over a network, not using a protocol which has implicit flow control like TCP. E.g. words like "the lexer was producing tokens too fast, so the parser applied backpressure" have never been heard.

Unix pipes have built in back pressure. A very simplified view, assuming blocking calls and single threads: if the process on the left side of the pipe produces data too fast and fills up the buffer, the operating system won’t give it any additional CPU cycles, until the program on the right side of the pipe caught up with reading and freed up the buffer. Things become more complicated when multi-threading and / or a…

Yes. Generally anything with fixed size queues provides back pressure.

And unbounded queues are a fiction.

Re: Backpressure explained – the resisted flow of data through software (2019)

#43

Post by jimmySixDOF » Wed, Mar 27, 2024, 2:57 PM UTC

ninkendo wrote: post #38

This is a strange article because it doesn’t even mention the simplest and most common form of backpressure, which is to make requests synchronous. ie. what you see in TCP and standard Unix sockets/pipes. The article makes it seem like you have three options: (1) Buffer, (2) drop, or (3) “control” the producer (and gives examples of “telling” the producer to slow down.) But the simplest thing to do is for a producer…

TCP Windowing is not exactly out of band signaling but is used to adapt sender receiver patterns through negotiation it looks a round trip times and adjusts accordingly which is a form of backpressure when its in a closing the window mode.

Re: Backpressure explained – the resisted flow of data through software (2019)

#44

Post by _3u10 » Wed, Mar 27, 2024, 4:03 PM UTC

luibelgo wrote: post #28

_3u10 wrote: post #3

Earlier quoted context omitted.

Use a LIFO with a timeout. When you find a request that exceeds the timeout clear the lifo.

why lifo? can you elaborate?

It stands for last in first out.

If you use a linked list it’s like always adding at the head of the list and always removing from the head.

Let’s say you have two clients for your server one sends a reliable 1 request per second, and the other sends 10,000 requests every hour in a burst.

The lifo will basically result in the 1/sec client always getting their requests answered and when you get a burst from the other client most of their requests will get dropped. Assume your server can handle 100 reqs/second with a 1 second time out.

Re: Backpressure explained – the resisted flow of data through software (2019)

#45

Post by pixelfarmer » Wed, Mar 27, 2024, 4:17 PM UTC

Darkstryder wrote: post #39

heavenlyblue wrote: post #34

Earlier quoted context omitted.

Why is jitter important in a queue?

To avoid something called the thundering herd problem: https://en.m.wikipedia.org/wiki/Thundering_herd_problem For instance, a bunch of clients all make a request to a server at the same time, briefly saturating the server. If all the clients have the same timeout without jitter, they will all try again together at the same time once the timeout expires, saturating the server again and again. Jitter helps by « spread…

The basic idea behind that is also used in all sorts of networks where you have multiple stations sharing the same medium with everyone being able to freely send stuff. To solve this, if a "collision" is detected, stations then use a random timeout before they send again in the hope that the next time there won't be another collision.

https://en.wikipedia.org/wiki/Carrier-sense_multiple_access_...

Re: Backpressure explained – the resisted flow of data through software (2019)

#46

Post by toast0 » Wed, Mar 27, 2024, 4:18 PM UTC

tootie wrote: post #5

Isn't the ideal solution to make the throttled system faster? Like autoscale horizontally and/or vertically, sharding or just writing better code? Everything in this article is about to cope with back pressure but solving is frequently possible.

In some ways, yes, a solution to your system is throughput constrained is to remove the constraint.

But, there's a cost to that, it may be development time, capital, or operational expense. Certainly, sometimes you spend a few minutes replacing a bad sort with a much better sort and get immediate and large benefits. But often it's the case where rare conditions cause loads that are too expensive to handle immediately.

Having backpressure setup and monitored in advance also comes at a cost, but allows you to make specific choices about how the system works in overload, and allows you to monitor the overload conditions, and hopefully/maybe gives you some feedback about your overall capacity.

For example, many systems have worse throughput when overloaded. Having a strict concurrency limit prevents throughput loss from context switching, and the errors (or simply effects from queueing) can propigate as backpressure. Monitoring active threads gives a sense of available capacity, monitoring time spent with threads at max or depth of queue / number of requests rejected gives a sense of unmet demand.

Re: Backpressure explained – the resisted flow of data through software (2019)

#47

Post by [deleted] » Wed, Mar 27, 2024, 5:12 PM UTC

vrosas wrote: post #18

jcgrillo wrote: post #10

Earlier quoted context omitted.

I'll bet your backpressure mechanism can react at least an order of magnitude faster than your scaling mechanism.

Serverless systems are pretty decent at scaling quickly these days. The problem is rarely lack of servers in my experience, though. You usually run out of database connections or some other bottleneck first.

[deleted]