Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Background job queues and priorities may be the wrong path

alexis.bernard.io

Post reply on HN
Print view View original

41–50 of 52 posts

Re: Background job queues and priorities may be the wrong path

#41

Post by deredede »

andrelaszlo wrote: post #24

Another important aspect that seems to be overlooked in a lot of these discussions (specifically for Rails, but probably relevant in other contexts as well) is that if your jobs are very heterogeneous in terms of their degree of parallelizability then it will be very difficult to do effective provisioning and capacity planning. Let's say you have two types of jobs: one that is highly parallelizable (Eg request to thi…

Do not use the same queues for I/O jobs and compute jobs, they are not competing for the same resources.

Re: Background job queues and priorities may be the wrong path

#42

Post by Dylan16807 »

arakageeta wrote: post #22
karmakaze wrote: post #15

Earlier quoted context omitted.

Yeah, I don't see how deadlines/SLOs does any better when there's insufficient capacity to meet those of queued jobs at their concurrency. > I don’t think it’s possible to meet a latency target through prioritization when there is a fundamental lack of capacity. This seemingly implies that it would be achievable with target deadlines/SLOs. At capacity I don't see a better solution than give each priority (or target l…

Under deadline scheduling, every pending job _eventually_ has highest priority as time elapses. (Assuming new jobs can’t arrive with deadlines in the past.) Every job is eventually serviced. The “pain” experienced in an overload situations is spread among all late jobs. Contrast this with fixed-priority scheduling, where lowest priority jobs will be starved completely, until the overload is resolved.

> The “pain” experienced in an overload situations is spread among all late jobs. Contrast this with fixed-priority scheduling, where lowest priority jobs will be starved completely, until the overload is resolved.

Though this is often not a good way to spread out the pain.

It's probably much worse for the 15s job to be a minute late than for the 8h job to be a minute late, but basic deadline scheduling will treat them the same. So you want sharing, but uneven sharing.

Re: Background job queues and priorities may be the wrong path

#43

Post by dehrmann »

paulsutter wrote: post #8

If you want to write any complicated concurrent code, the simplest and best way is one long polling loop state machine. You might use a thread to call a blocking API, but the majority of the logic should be in a polling loop. I used to love chained callbacks when I was 16, and later I thought threads were the greatest, and I've written a bunch of device drivers that operate at different IPLs. But 20 years ago a cofou…

Half of what you're describing is an async event loop, but it isn't strictly tied to a state machine.

Re: Background job queues and priorities may be the wrong path

#44

Post by okeuro49 »

jupp0r wrote: post #11
paulsutter wrote: post #8

If you want to write any complicated concurrent code, the simplest and best way is one long polling loop state machine. You might use a thread to call a blocking API, but the majority of the logic should be in a polling loop. I used to love chained callbacks when I was 16, and later I thought threads were the greatest, and I've written a bunch of device drivers that operate at different IPLs. But 20 years ago a cofou…

Ideally, you don't have blocking APIs at all. There isn't any reason nowadays. Every blocking API in your system is there for historic reasons (ie it's >20 years old) or because somebody made a terrible design decision in the last 20 years. I am obviously referring to operations that do actual IO or wait for work on other threads only.

> Ideally, you don't have blocking APIs at all. There isn't any reason nowadays.

If you're working with the JVM, you can write simpler blocking code that is also performant through virtual threads.

Re: Background job queues and priorities may be the wrong path

#45

Post by F-W-M »

paulsutter wrote: post #8

If you want to write any complicated concurrent code, the simplest and best way is one long polling loop state machine. You might use a thread to call a blocking API, but the majority of the logic should be in a polling loop. I used to love chained callbacks when I was 16, and later I thought threads were the greatest, and I've written a bunch of device drivers that operate at different IPLs. But 20 years ago a cofou…

What do you mean by a long polling state machine? Is it just a state machine that gets fed via long polling http requests?

Re: Background job queues and priorities may be the wrong path

#46

Post by akvadrako »

F-W-M wrote: post #45
paulsutter wrote: post #8

If you want to write any complicated concurrent code, the simplest and best way is one long polling loop state machine. You might use a thread to call a blocking API, but the majority of the logic should be in a polling loop. I used to love chained callbacks when I was 16, and later I thought threads were the greatest, and I've written a bunch of device drivers that operate at different IPLs. But 20 years ago a cofou…

What do you mean by a long polling state machine? Is it just a state machine that gets fed via long polling http requests?

No, he means a run loop which polls for events, often by using poll() or select(), but it could also just be sleep() and then a SELECT statement.

Then it contains logic to handle events.

Re: Background job queues and priorities may be the wrong path

#48

Post by dudeinjapan »

deredede wrote: post #40
dudeinjapan wrote: post #39

What is missing from this picture is idleness. For example, suppose I have a SLO 10 sec job A and SLO 5 min job B. If I only get a few Bs sporadically, I may want to define queue X=A only, and queue Y=A,B to use the idle compute to process more As. In the wild, this is a delicate balancing act.

Even if you get lots of Bs, if B takes 15s to run, and you ever get n_worker requests for B at the same time immediately followed by a single A, you lose, even with plenty capacity to spare. You need either dedicated workers for low latency tasks or some sort of preemption to meet SLOs with such heterogeneous tasks.

Right you need to think about the queue wait time you are willing to tolerate, in addition the time it takes a job to run. If you arent willing to wait in queue then you will need to have idle capacity.

Re: Background job queues and priorities may be the wrong path

#49

Post by F-W-M »

akvadrako wrote: post #46
F-W-M wrote: post #45

Earlier quoted context omitted.

What do you mean by a long polling state machine? Is it just a state machine that gets fed via long polling http requests?

No, he means a run loop which polls for events, often by using poll() or select(), but it could also just be sleep() and then a SELECT statement. Then it contains logic to handle events.

Ah thanks. Didn't hear 'long polling' in that context yet.

Re: Background job queues and priorities may be the wrong path

#50

Post by andrelaszlo »

deredede wrote: post #41
andrelaszlo wrote: post #24

Another important aspect that seems to be overlooked in a lot of these discussions (specifically for Rails, but probably relevant in other contexts as well) is that if your jobs are very heterogeneous in terms of their degree of parallelizability then it will be very difficult to do effective provisioning and capacity planning. Let's say you have two types of jobs: one that is highly parallelizable (Eg request to thi…

Do not use the same queues for I/O jobs and compute jobs, they are not competing for the same resources.

Totally agree.
Post reply on HN