Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Thundering Herd Problem

encore.dev

Post reply on HN
Print view View original

31–38 of 38 posts

Re: The Thundering Herd Problem

#31

Post by Rapzid »

PaulDavisThe1st wrote: post #18

Once again, what TFA describes is NOT the thundering herd problem. https://en.wikipedia.org/wiki/Thundering_herd_problem > In computer science, the thundering herd problem occurs when a large number of processes or threads waiting for an event are awoken when that event occurs, but only one process is able to handle the event . (emphasis mine)

Okay but that's actually not the most general definition I'm familiar with.

It's a synchronization and resource constraint issue with various symptoms depending on domain specifics.

Let's discuss specifics about how the post isn't discussing a thundering heard problem in the general sense..

Re: The Thundering Herd Problem

#32

Post by PaulDavisThe1st »

Rapzid wrote: post #31
PaulDavisThe1st wrote: post #18

Once again, what TFA describes is NOT the thundering herd problem. https://en.wikipedia.org/wiki/Thundering_herd_problem > In computer science, the thundering herd problem occurs when a large number of processes or threads waiting for an event are awoken when that event occurs, but only one process is able to handle the event . (emphasis mine)

Okay but that's actually not the most general definition I'm familiar with. It's a synchronization and resource constraint issue with various symptoms depending on domain specifics. Let's discuss specifics about how the post isn't discussing a thundering heard problem in the general sense..

Please read the comment from user gilbetron appearing really close to yours, and my reply.

Re: The Thundering Herd Problem

#33

Post by Rapzid »

PaulDavisThe1st wrote: post #32
Rapzid wrote: post #31

Earlier quoted context omitted.

Okay but that's actually not the most general definition I'm familiar with. It's a synchronization and resource constraint issue with various symptoms depending on domain specifics. Let's discuss specifics about how the post isn't discussing a thundering heard problem in the general sense..

Please read the comment from user gilbetron appearing really close to yours, and my reply.

Ah sorry, didn't realize thundering heard was solved by newer kernels. I can remove all my exponential back off with jitter retry strategies.

Re: The Thundering Herd Problem

#34

Post by PaulDavisThe1st »

Rapzid wrote: post #33
PaulDavisThe1st wrote: post #32

Earlier quoted context omitted.

Please read the comment from user gilbetron appearing really close to yours, and my reply.

Ah sorry, didn't realize thundering heard was solved by newer kernels. I can remove all my exponential back off with jitter retry strategies.

Your exponential back offs were not a solution to the problem identified and named in the mid-90s. They might be valuable for some other problem.

Re: The Thundering Herd Problem

#35

Post by pcthrowaway »

oops wrote: post #27
01HNNWZ0MV43FF wrote: post #23

Earlier quoted context omitted.

I recall seeing that called "Request coalescing" in Faster Than Lime's article https://fasterthanli.me/articles/request-coalescing-in-async...

It is also sometimes called request collapsing.

It's also sometimes called request deduplication

Re: The Thundering Herd Problem

#36

Post by pcthrowaway »

PaulDavisThe1st wrote: post #22
gilbetron wrote: post #20

Earlier quoted context omitted.

Thundering herd has evolved beyond the original definition, that happens. Sometimes terms even flip their meaning (like how "one bad apple" has flipped it's meaning, or how "pre-optimization is the root of all evil" as used these days isn't what the original author actually meant, but in this case I think most of the new usages fit the basic idea, which is a large amount of unusual traffic happens outside of normal u…

Fair point that terms like this can change meanings. But this version absolutely does not fit the original, which had nothing to do with amounts of traffic. It was a problem with kernel APIs: if N threads sleep waiting for an event on a socket (i.e. a thread pool), there was no way to wake up only one of them when a single msg arrived. They would all wake, all check if there was work to do, then all (but one) would g…

I initially heard of "thundering herd" in reference to the pattern where a server handling N requests for the same data may only need to run the data handling logic once to fulfill all of those requests.

For example, if I have a server with an endpoint that needs to make a request to a different service for some data, I don't want to make that request 10 times when my server receives 10 requests while the first request is being handled; all 10 of those incoming requests can be fulfilled by 1 outgoing request to the secondary service.

In that sense, it's very similar to what you described, but it's still likely one process handling the requests.

I'll admit that the author seemed to use "thundering herd" in reference to your server just suddenly receiving a lot of traffic, which is also different from the usage I was familiar with.

Re: The Thundering Herd Problem

#37

Post by Rapzid »

PaulDavisThe1st wrote: post #34
Rapzid wrote: post #33

Earlier quoted context omitted.

Ah sorry, didn't realize thundering heard was solved by newer kernels. I can remove all my exponential back off with jitter retry strategies.

Your exponential back offs were not a solution to the problem identified and named in the mid-90s. They might be valuable for some other problem.

Good thing the problem is solved then.

Re: The Thundering Herd Problem

#38

Post by btbuildem »

Wouldn't the obvious precaution here be to roll out the new feature to just a portion of the waitlist?

I guess the risk here is that you'd never hit the same thresholds as when letting it all rip at once, but wouldn't it be a bit more graceful and let you work out at least some of the problems in front of a smaller audience?

Post reply on HN