Live data from Hacker News

Job queues are deceptively tricky

typesanitizer.com

51–59 of 59 posts

Re: Job queues are deceptively tricky

#51

Earlier quoted context omitted.

Bad take. Most of the different behaviors you criticize about message queue systems arise from them needing to a) distribute work b) durably, c) over the network. Most message queues want to be able to distribute work to multiple parallel consumers, either for performance, redundancy, re-deployment of the consumers, and so on. Like, sure, you can do that with named pipes, but they're not durable; their state can be l…

I just talked about pipes as a useful conceptual tool. You’re assuming what applications I think they apply to. But I’ll do my best. One of my takeaways is that if you do not handle on data transfer in the http request/response then you are choosing to exit the pipe model. Once you do that, yes all kinds of crazy async problems exist which require very complex tools to wrangle. See the sibiling comment from Google en…

> The answer is usually a performance problem for which this is a bandaid.

I agree that if that's the problem you're trying to solve, MQs are a poor stop-gap and not a general-purpose solution.

But many (most?) MQs aren't deployed for that reason; rather, they're used to either a) defer work whose latency characteristics are incompatible with the producer's runtime (e.g. sending email which might take minutes/hours to be accepted upstream can't be done synchronously from an HTTP request handler), b) handle work which doesn't need to be done transactionally and which has a high probability of needing to connect to resources whose uptime you don't control (queues make retries easy), or c) work that needs to be batched or otherwise completed on a schedule or dimensionality that's not available in the producer.

> A while loop blocked on a socket read with a producer with a write does not generate any extra network “chatter”.

True, and some MQs support that pattern, but it's rare for the same reason that most network server applications' core loop doesn't wrap a blocking read. While-blocking-read plays poorly with timeouts, restarts, and multiplexing/wait-any. This isn't unique to MQs.

> Anytime you are using callbacks you are choosing not to have a thread with imperative logic.

I mean, it's pretty easy in most languages/runtimes to turn a callback into an imperative wait. When it comes to network servers specifically, I think exposing the lowest-runtime-overhead model as the primary API (selectors and callbacks) and letting users add control flow primitives on top of that is preferable, but reasonable minds can differ here. This also isn't unique to MQs.

> if you want to send data between two systems and you have producers and consumers at different rates, you don’t have to do anything special. The UNIX kernel is designed to solve this problem.

For ephemeral/can-be-lost local traffic, you're partly right (pipes/fifos are a simple and widely available tool that can help with this, though you have to layer some protocol-ish logic on top to make them work reliably between consumers that expect particular structures and don't speak ASCII/newlines alone). For durable local traffic, or remote traffic, UNIX doesn't have a prebuilt primitive. TCP tx/rx window sizes aren't surfaced such that buffers and backpressure can be reliably interpreted by userland to make delivery decisions.

Re: Job queues are deceptively tricky

#52
post #37

Earlier quoted context omitted.

To explain this: if the system is 95% utilized, and a new request comes in, there's a 95% chance the system is already busy and the request has to wait. But after the currently in-progress request finishes, there's still a 95% chance the system is busy (with another request that was already queued behind it) and request X has to wait. After that one finishes, same thing. On average, request X has to wait for about 20…

Your math doesn't make any sense.

Which part? 95% utilisation means at any random point in time there's a 95% chance the system is utilized and a 5% chance it is not utilized at that moment, so on average you have to try 20 times to find an unutilised moment?

Re: Job queues are deceptively tricky

#53

Earlier quoted context omitted.

I just talked about pipes as a useful conceptual tool. You’re assuming what applications I think they apply to. But I’ll do my best. One of my takeaways is that if you do not handle on data transfer in the http request/response then you are choosing to exit the pipe model. Once you do that, yes all kinds of crazy async problems exist which require very complex tools to wrangle. See the sibiling comment from Google en…

> The answer is usually a performance problem for which this is a bandaid. I agree that if that's the problem you're trying to solve, MQs are a poor stop-gap and not a general-purpose solution. But many (most?) MQs aren't deployed for that reason; rather, they're used to either a) defer work whose latency characteristics are incompatible with the producer's runtime (e.g. sending email which might take minutes/hours t…

Thanks for high quality reply.

> latency characteristics are incompatible with the producer's runtime

> handle work which doesn't need to be done transactionally and which has a high probability of needing to connect to resources whose uptime you don't control (queues make retries easy

I agree. I think you do need to break the request/response pipe for this use case.

I traditionally have treated this as a state in a database. I know there are limitations and I can see MQ being a solution. I want to try the email idea sometime.

I do think queues are a poor technology for dealing with performance problems in a web stack.

> This also isn't unique to MQs.

Agreed. My ranting about async and callbacks is an ongoing project.

I think pipes solve producer/consumer problems in a way most engineers don’t appreciate.

> While-blocking-read plays poorly with timeouts, restarts, and multiplexing/wait-any.

The primary problem solved by not using blocking primitives is to try to free up OS resources from threads (green threading). Why would an internal queue have that problem? It’s not accepting arbitrary connections.

Re: Job queues are deceptively tricky

#54

Earlier quoted context omitted.

Properly scaling queue consumers is a problem I've spent a lot of time on in the last few years. Working on a messaging platform with highly variable traffic, including close to zero during the night, means that capacity provisioning according to the max will be very costly, and lead to a lot of frustration when you are saturated anyway. Indeed you need backpressure but the traditional methods (CPU usage or similar m…

> capacity provisioning according to the max will be very costly I’m skeptical. You can support a pretty massive messaging system with one box. What did you try? What went wrong?

Similar to what gp mentioned, max latency. You can tune your consumers to burn through the queue and push messages, but when you have latency constraints e.g. a very tight deadline the first few messages in the queue might achieve it but not the 20000th.

When you want to switch from no processing to say 2M messages than need to be sent within 5 minutes it's difficult to appropriately provision without fast scale out.

Re: Job queues are deceptively tricky

#55

Earlier quoted context omitted.

> The answer is usually a performance problem for which this is a bandaid. I agree that if that's the problem you're trying to solve, MQs are a poor stop-gap and not a general-purpose solution. But many (most?) MQs aren't deployed for that reason; rather, they're used to either a) defer work whose latency characteristics are incompatible with the producer's runtime (e.g. sending email which might take minutes/hours t…

Thanks for high quality reply. > latency characteristics are incompatible with the producer's runtime > handle work which doesn't need to be done transactionally and which has a high probability of needing to connect to resources whose uptime you don't control (queues make retries easy I agree. I think you do need to break the request/response pipe for this use case. I traditionally have treated this as a state in a…

> Thanks for high quality reply.

Likewise; thanks for engaging constructively as well.

> I traditionally have treated this as a state in a database.

Which is a super appropriate tool many (most) times! I've been using "MQ" in this context refers to the conceptual capability to externalize/distribute/persist work with a push/pop API. That can be provided by a database, a UNIX pipe, or a more traditional message broker--the reasons-to-adopt and costs/benefits are largely the same.

> I want to try the email idea sometime.

Email (and the print spool) are some of the oldest message-queue primitives on UNIX systems, I think? I'm not sure if POSIX/shmem MQs predate them or not, but they're all quite venerable proofs that the pattern has its uses. I'm sure you can (maybe others already have?) use SMTP and the mailq to manage application-internal queue communication. I'm not sure how low-overhead or performant it'll be given that it's highly optimized for one/very few consumers and delivery attempts >1 being attempted on a pretty large time scale compared to application traffic, but it'd be an interesting experiment to be sure!

Re: Job queues are deceptively tricky

#56
post #42

The UNIX pipe really is an incredible concurrency invention which is not well understood, and attempts to work around its features turn into bugs. A buffer to accumulate data that blocks when it’s full allows you to handle bursty loads. It solves the back pressure problem of readers and writers operating at different speeds. It doesn’t over consume resources. It also solves the architecture problem of when to trigger…

The UNIX pipe has the (for many systems) undesirable negative property of losing data in the pipe when the receiving process terminates: Anything in the kernel buffer of the pipe gets lost. Since the pipe is generally unidirectional, the sending process has no way of knowing whether the receiving process has received, or even more successfully processed, anything sent. For that, one needs to make a pipe in the opposi…

> The UNIX pipe has the (for many systems) undesirable negative property of losing data in the pipe when the receiving process terminates: Anything in the kernel buffer of the pipe gets lost.

There are solutions for that, though:

* You can observe and/or persist intermediate data with `tee`

* You can use a named pipe, whose lifetime is bound to an inode instead of the processes

* You can dup the read end of the pipe and pass the extra file descriptor to a crash handler process

None of these are esoteric. You do need a basic understanding of Unix primitives, but any half-decent computer science course will cover them.

Re: Job queues are deceptively tricky

#57
post #29
post #7

I think the the second part was kinda obvious? The moment I read this: > If you’re anything like me, you would probably have said Parallel Spawn, Prefer New, and Wait are perhaps defensible, whereas Prefer Old feels weird/backward. it was pretty obvious I was not anything like him. My intuitive answers are pretty different. - Parallel Spawn is useful, but it's orthogonal to the rest. Even if you have 4 workers, you'l…

> - "Prefer New" seems almost useless. You've already spent all this effort doing the job, why are you cancelling it and throwing away the results? The obvious use-case is if the result depends on a state and the state has changed since the job has started. Examples would be updating the landing page when a new article has come in (you don't need the outdated landing page w/o the new article) or if you did a code cha…

For external events, like code update or new publish, sure. But the author talks about scheduled events - it's harder to think of good examples here.

(because with scheduled events, if you missed deadline once, there is a good chance that restarting with exactly the same deadline will miss again, and again, and you will never finish. Better do "prefer old" and at least have _some_ finished data)

Re: Job queues are deceptively tricky

#58
post #52

Earlier quoted context omitted.

Your math doesn't make any sense.

Which part? 95% utilisation means at any random point in time there's a 95% chance the system is utilized and a 5% chance it is not utilized at that moment, so on average you have to try 20 times to find an unutilised moment?

No, none of that is true.

Think about a secretarial pool with five secretaries. The pool is at 95% utilization. This might mean that one of the secretaries is at 100% utilization and the other four are at 93.75%. It might mean that all of the secretaries are at 95%. It might mean that four of them are at 100% and one is at 75%. It might mean something else.

Going back to that third case, we can easily see that with the system at 95% utilization, at any given moment in time there is a 75% chance that the system is busy and unable to begin a new task, and a 25% chance that the system has capacity to begin a new task right away.

In the full-average case where all five secretaries are at 95% utilization, the odds that the system has spare capacity at any given moment are 22.62% (= 1 - 0.95 ^ 5), not too different from the case where four secretaries are kept busy 100% of the time.

Post reply on HN