Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why disaster happens at the edges: An introduction to queue theory

thenewstack.io

Post reply on HN
Print view View original

41–50 of 62 posts

Re: Why disaster happens at the edges: An introduction to queue theory

#41

Post by ivalm »

btilly wrote: post #29
gizmo686 wrote: post #28

Earlier quoted context omitted.

The moral of that story is don't let your queue go up to infinity. Drop 1/101 events and serve the other 99% with a typical latency. Instead of latency going to infinity to everyone, your service gracefully-ish degrades by serving as many requests as it is able to and quickly reporting an error to those it is not. Once load drops below capacity, you are immedietly back to a fully functioning state. Granted, depending…

So we should solve the problem at the port of Long Beach by randomly dropping cargo into the ocean until we no longer get container ships piling up off the coast?

I know you mean it sarcastically, but I am guessing this could solve the problem… the throughput of the ports would increase and the demand shock ameliorated.

Re: Why disaster happens at the edges: An introduction to queue theory

#42

Post by jimmySixDOF »

jph wrote: post #7

Queueing theory-- my notes for programmers: https://github.com/joelparkerhenderson/queueing-theory Queueing theory is the mathematical study of waiting lines, or queues. We use queueing theory in our software development, for purposes such as project management kanban boards, inter-process communication message queues, and devops continuous deployment pipelines.

All kinds of Queue analysis is also done extensively in Six Sigma practice

Re: Why disaster happens at the edges: An introduction to queue theory

#43

Post by whatever1 »

roenxi wrote: post #14

Queue theory is almost a distraction because it overcomplicates the situation. The underlying insight is that rates matter. If a system can handle 100 events a second, then 99 events per second everything is fine and 101 events per second the system is down. There is a threshold where everything falls apart. Queue theory encourages people to think in terms of flow rates and has the technically correct de-rates to nom…

The main question posed to business is the following.:

Do you want to provision your system for the worst case scenario volume, and what is this worst case scenario you are willing to pay to insure your system against.

It’s an easy question, but I don’t see how queueing theory can provide answers to. It is at its root a subjective business decision.

Re: Why disaster happens at the edges: An introduction to queue theory

#44

Post by kqr »

NegativeLatency wrote: post #27
tharkun__ wrote: post #21

Earlier quoted context omitted.

Why is 101 events per second == system down? Of course there are systems where not being able to process everything right when it comes in means that you're "down". It's not a given though and queuing systems are actually perfect for use cases where not being able to process at the speed of incoming requests is completely fine. Eventual consistency is a thing. I have the same experience though that it seems to be har…

Assuming you have queues and stuff that can hold the requests in the intermediary time. Some systems will just start dropping the data or crashing and loose data. If you have a queue or something in the middle then the failure is potentially recoverable or never happens in the first place.

Queues don't fix overload: https://ferd.ca/queues-don-t-fix-overload.html

Re: Why disaster happens at the edges: An introduction to queue theory

#45

Post by vishnugupta »

kqr wrote: post #40
justicezyx wrote: post #35

Earlier quoted context omitted.

Another random software engineer claiming a fundamental theoretical problem irrelevant... Queuing theory deals with the stochastic behavior of a discreet event in at a single queue, and possibly multiple ones connected in some structured manner. It's importance is that it can determine before hand what are the stochastic boundary of the system. Your whole idea of 99 events and 101 events are so strawman that I don't…

But things do have an average rate. And in my experience, this is very practical information. Almost daily I plug mean rates into Little's law and get useful numbers out. (Yes, I do verify my predictions.)

> Little's law

I guess that's the OP's point that you can't escape from talking about queues and which you sort of validated by bringing up Little's law.

Queues are literally everywhere around us in the physical world. It needs to be understood at a basic level -- Little's law is good enough and only requires primary school math. If execs aren't able to understand such a ubiquitous and simple concept then I don't know what to say.

Average flow rate is indeed useful but please explain that along with a queue and show whoever needs to know how queues can grow unboundedly if output rate is consistently less than input rate. And if needed explain further the upstream consequences of unbounded queue growth.

Re: Why disaster happens at the edges: An introduction to queue theory

#46

Post by marius_k »

kqr wrote: post #44
NegativeLatency wrote: post #27

Earlier quoted context omitted.

Assuming you have queues and stuff that can hold the requests in the intermediary time. Some systems will just start dropping the data or crashing and loose data. If you have a queue or something in the middle then the failure is potentially recoverable or never happens in the first place.

Queues don't fix overload: https://ferd.ca/queues-don-t-fix-overload.html

It lowers the load if the consumer is able to process batches of requests efficiently (ex. for vector operations).

Re: Why disaster happens at the edges: An introduction to queue theory

#47

Post by antman »

roenxi wrote: post #14

Queue theory is almost a distraction because it overcomplicates the situation. The underlying insight is that rates matter. If a system can handle 100 events a second, then 99 events per second everything is fine and 101 events per second the system is down. There is a threshold where everything falls apart. Queue theory encourages people to think in terms of flow rates and has the technically correct de-rates to nom…

That is absolutely not what queuing theory says. In equally simplified terms, queuing theory says that for a system that handles 99 events per second if you size it for 100 it will probably fail because 99 is just the mean, and when you overshoot (in events per second) you don’t have the capability to cover it during the undershoot (in events per second) period because events have piled up.

For a numerical example of queuing theory 101 and why variance is very important rather than something that “scares people away” [0]. Flow tracking is not enough.

Also this discussion is relevant [0]:https://www.johndcook.com/blog/2008/10/21/what-happens-when-...

Re: Why disaster happens at the edges: An introduction to queue theory

#48

Post by vosper »

antman wrote: post #47
roenxi wrote: post #14

Queue theory is almost a distraction because it overcomplicates the situation. The underlying insight is that rates matter. If a system can handle 100 events a second, then 99 events per second everything is fine and 101 events per second the system is down. There is a threshold where everything falls apart. Queue theory encourages people to think in terms of flow rates and has the technically correct de-rates to nom…

That is absolutely not what queuing theory says. In equally simplified terms, queuing theory says that for a system that handles 99 events per second if you size it for 100 it will probably fail because 99 is just the mean, and when you overshoot (in events per second) you don’t have the capability to cover it during the undershoot (in events per second) period because events have piled up. For a numerical example of…

I don’t know anything about queue theory, but I’m not sure how your point differs from GP? You both said that a system designed for 100 and being run at 99 falls apart at 101?

I didn’t understand GPs comment about flows - perhaps that’s where the dispute is.

Genuine question, looking forward to learning more :)

Re: Why disaster happens at the edges: An introduction to queue theory

#50

Post by jph »

layer8 wrote: post #24
jph wrote: post #7

Queueing theory-- my notes for programmers: https://github.com/joelparkerhenderson/queueing-theory Queueing theory is the mathematical study of waiting lines, or queues. We use queueing theory in our software development, for purposes such as project management kanban boards, inter-process communication message queues, and devops continuous deployment pipelines.

The notes don’t explain what problems queue theory solves, i.e. why would I want to use it, as a programmer. The Wikipedia article’s second sentence is a bit more helpful in that respect: "A queueing model is constructed so that queue lengths and waiting time can be predicted."

We typically track many things about the activities in the queue, and we want to summarize the results by choosing a shortlist of the most relevant ones for our projects.

    Dμ = Delivery service rate. Devops teams may say "deployment frequency" or "we ship X times per day".

    Rτ = Restore lead time. Site reliability engineers may say "time to restore service" or "mean time to restore (MTTR)".
Post reply on HN