TheNewStack needs more articles like this one and less articles that talk about some new feature where by adding extra five lines of YAML you get something or other in your Kubernetes.
Why disaster happens at the edges: An introduction to queue theory
11–20 of 62 posts
Re: Why disaster happens at the edges: An introduction to queue theory
#12A frustrating headline. Where else would it happen? The middle? Crystals fracture on their faces. Things tend to break along their boundaries.
Astroworld Festival was a mainstream festival with average aged festival goers dying in the main act.
I have no idea on your point? A lot of people are trying to make Astroworld be on a boundary, I guess?
Step one you worry about the middle. If crystals don't also fracture on the middle then it's a bad analogy. Rocks aren't the same as processes.
Re: Why disaster happens at the edges: An introduction to queue theory
#13A frustrating headline. Where else would it happen? The middle? Crystals fracture on their faces. Things tend to break along their boundaries.
I feel like you are understanding the title as I first understood it, meaning like, "the external facing portions of infrastructure". However, reading the article, it seems clear that he's referring to the edges of a distribution curve (i.e. Infrequent events that impact experience nonetheless).
From the article: "It’s tempting to focus on the peak of the curve. That’s where most of the results are. But the edges are where the action is. Events out on the tails may happen less frequently, but they still happen. In digital systems, where billions of events take place in a matter of seconds, one-in-a-million occurrences happen all the time. And they have an outsize impact on user experience."
EDIT: IMO, the title is still a little annoying in this respect. I think everyone would agree if a request to your site fails 5% of the time, that is unacceptable, even though it "usually works." The discussion of the distribution curve simply to make the point that spikes in usage cause backed up queues which impact performance isn't necessarily helpful as far as I can tell, and it seems done largely in service to the title. In my mind while reading this, I'm thinking, "Okay, cool, but how does the fact that this interesting issue exists at the edge of the curve help me identify it?" Answer: It doesn't. If you see errors occurring, you will investigate them once they are noticed. Being at the edge of the curve may mean it takes longer to notice, but like, what kind of alerting system are you using that discriminates against rare issues in favor of common ones?
Discussing queues, over provisioning, back pressure, etc. are all super interesting and helpful.
Re: Why disaster happens at the edges: An introduction to queue theory
#14Queue theory is almost a distraction because it overcomplicates the situation. The underlying insight is that rates matter. If a system can handle 100 events a second, then 99 events per second everything is fine and 101 events per second the system is down. There is a threshold where everything falls apart.
Queue theory encourages people to think in terms of flow rates and has the technically correct de-rates to nominal capacity to account for variance. But normally that is overkill except in highly engineered systems.
The battle in an organisation is convincing people to look at flows at all. In my experience people love metrics that track stock (we have X widgets or can handle Y orders) and not flows (we built A widgets and sold B widgets). Anyone who has training in queue theory knows when it is appropriate to look at flows rather than stocks and that is where the value is. The formulas and variances tend to just scare people away from monitoring flows because they don't understand what is happening.
Re: Why disaster happens at the edges: An introduction to queue theory
#15Queue theory is almost a distraction because it overcomplicates the situation. The underlying insight is that rates matter. If a system can handle 100 events a second, then 99 events per second everything is fine and 101 events per second the system is down. There is a threshold where everything falls apart. Queue theory encourages people to think in terms of flow rates and has the technically correct de-rates to nom…
> The underlying insight is that rates matter. If a system can handle 100 events a second, then 99 events per second everything is fine and 101 events per second the system is down.
Not really. You need actual theory to inform you whether 99 events per second is okay. See https://www.johndcook.com/blog/2008/10/21/what-happens-when-...
Re: Why disaster happens at the edges: An introduction to queue theory
#16I worked at IBM in earlier days, and a couple of the Research people in the data storage group at Almaden and Poughkeepsie gave some great lectures and made excellent tools for modeling cache disk storage array performance that were based heavily on queue theory. This brings back great memories, thanks!
Re: Why disaster happens at the edges: An introduction to queue theory
#17Queue theory is almost a distraction because it overcomplicates the situation. The underlying insight is that rates matter. If a system can handle 100 events a second, then 99 events per second everything is fine and 101 events per second the system is down. There is a threshold where everything falls apart. Queue theory encourages people to think in terms of flow rates and has the technically correct de-rates to nom…
> The battle in an organization is convincing people to look at flow at all.
I guess that's the point... an organization needs enough people to know about queue theory "to look at flow" and understand that queues are ubiquitous and anticipate that these queues experience a latency phase change at some point depending on utilization (and job duration variance). If an organization doesn't have anyone who understands queue theory, then it's probably more likely to have queue-related failures.
Re: Why disaster happens at the edges: An introduction to queue theory
#18There are secondary non-obvious effects of caching and queueing.
Cold start is an important scenario that lots of people overlook - “we won’t restart it during busy times”, etc. however if you have a bug or outage then that might be precisely when you restart. During a cold start you will often find your system behaves as it does at the tails of the performance curve, either because you are getting 100% cache misses, or because all your queues are full due to a client pile-on or queued upstream work, or some combination.
The article described back pressure; I can’t emphasize that enough as part of any resilient queuing system. Chains of queues are only as performant as their weakest link.
Re: Why disaster happens at the edges: An introduction to queue theory
#19Throughout my career, the one thing I've noticed engineers/developers consistently have trouble modeling in their heads is queuing.
Re: Why disaster happens at the edges: An introduction to queue theory
#20Queue theory is almost a distraction because it overcomplicates the situation. The underlying insight is that rates matter. If a system can handle 100 events a second, then 99 events per second everything is fine and 101 events per second the system is down. There is a threshold where everything falls apart. Queue theory encourages people to think in terms of flow rates and has the technically correct de-rates to nom…
> The underlying insight is that rates matter. If a system can handle 100 events a second, then 99 events per second everything is fine and 101 events per second the system is down. Not really. You need actual theory to inform you whether 99 events per second is okay. See https://www.johndcook.com/blog/2008/10/21/what-happens-when-...
If you want to maintain 99.99989% uptime or are about to spend $10 million, yes. If you want the cheap option then measure (arrival rate / service rate) and try to keep that below 80%. Then if that doesn't work adjust accordingly. That isn't really queue theory because it doesn't consider variance and requires effectively no knowledge of literature, what rho means or why people use M/M/1 queues as a default. Don't have to spell Poisson. But it will usually get you to where you want to be.
Most of the time feeling it out in production is a lot cheaper than keeping an in house specialist. If it isn't then sure, maintain a specialist to monitor the situation. It is a relatively rare role though.