Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Concurrency Limits by Netflix

github.com

Post reply on HN
Print view View original

1–10 of 30 posts

Re: Concurrency Limits by Netflix

#2

Post by jsnell »

The blog post announcing this might be a better read:

https://medium.com/@NetflixTechBlog/performance-under-load-3...

(And while reading that and thinking "I swear I recently read something else talking about applying TCP congestion control to RPC queueing". And indeed I had: http://www.evanjones.ca/prevent-server-overload.html)

Re: Concurrency Limits by Netflix

#4

Post by cperciva »

jsnell wrote: post #2

The blog post announcing this might be a better read: https://medium.com/@NetflixTechBlog/performance-under-load-3... (And while reading that and thinking "I swear I recently read something else talking about applying TCP congestion control to RPC queueing". And indeed I had: http://www.evanjones.ca/prevent-server-overload.html )

I swear I recently read something else talking about applying TCP congestion control to RPC queueing

I did this almost 10 years ago for accessing Amazon SimpleDB, too: http://www.daemonology.net/blog/2008-06-29-high-performance-...

Re: Concurrency Limits by Netflix

#5

Post by ramchip »

This reminds me a lot of the sbroker library in Erlang:

> SBroker is a framework to build fault tolerant task schedulers. It includes several built in schedulers based on TCP (and other network scheduling) algorithms.

> [...] in an ideal situation a target queue time would be chosen that keeps the system feeling responsive and clients would give up at a rate such that in the long term clients spend up to the target time in the queue. This is sojourn (queue waiting) time active queue management. CoDel and PIE are two state of the art active queue management algorithms with a target sojourn time, so should use those with defaults that keep systems feeling responsive to a user.

http://www.erlang-factory.com/euc2017/james-fish

https://github.com/fishcakez/sbroker

Re: Concurrency Limits by Netflix

#6

Post by time0ut »

This library looks very interesting. I've used a similar approach for pulling batches of items from a queue (e.g. discover the optimal batch size and inter-poll wait time). There are plenty of other places we could benefit from something like this. I can't wait to try this out.

Netflix puts out some amazing Java libraries. I've had excellent results using Hystrix [0]. It has been an excellent addition to our systems.

[0] https://github.com/Netflix/Hystrix

Re: Concurrency Limits by Netflix

#7

Post by lkrubner »

I like the sounds of this:

"Executor -- The BlockingAdaptiveExecutor adapts the size of an internal thread pool to match the concurrency limit based on measured latencies of Runnable commands and will block when the limit has been reached."

I'm often surprised this kind of auto-scaling thread pool is not a more common thing in Java land.

Re: Concurrency Limits by Netflix

#8

Post by Twirrim »

cperciva wrote: post #4
jsnell wrote: post #2

The blog post announcing this might be a better read: https://medium.com/@NetflixTechBlog/performance-under-load-3... (And while reading that and thinking "I swear I recently read something else talking about applying TCP congestion control to RPC queueing". And indeed I had: http://www.evanjones.ca/prevent-server-overload.html )

I swear I recently read something else talking about applying TCP congestion control to RPC queueing I did this almost 10 years ago for accessing Amazon SimpleDB, too: http://www.daemonology.net/blog/2008-06-29-high-performance-...

Had a co-worker implement something similar to this dynamic throttling in a back-end analytical process, based around access of a DynamoDB table that was accessed by multiple front end services.

The code on the back-end processing software would throttle itself back hard (about 50% of the speed it had reached) whenever it ran into a DynamoDB throttle message as a response, and then would ramp itself back up steadily. Combined with good retry logic on the front end services, it meant we could keep the DynamoDB table humming along at near maximum usage.

Re: Concurrency Limits by Netflix

#9

Post by tyrankh »

lkrubner wrote: post #7

I like the sounds of this: " Executor -- The BlockingAdaptiveExecutor adapts the size of an internal thread pool to match the concurrency limit based on measured latencies of Runnable commands and will block when the limit has been reached. " I'm often surprised this kind of auto-scaling thread pool is not a more common thing in Java land.

Agreed - not trying to trivialize the work here, but this thing seems solved with tools at hand. Perhaps I miss some of the subtlety in their usecase, though.
Post reply on HN