Improving distributed caching performance and efficiency at Pinterest

Post by infocollector » Sun, Jul 03, 2022, 3:40 AM UTC

Post by infocollector » Sun, Jul 03, 2022, 5:45 AM UTC

dormando wrote: post #3

infocollector wrote: post #2

I do hope Pinterest would fund memcached development going forward, especially the language clients for memcached. I've been using pylibmc to access memcached fast, and that project seems to be almost dead ( https://github.com/lericson/pylibmc/issues ).

client ecosystem is definitely a sore point now. I've just sort of started working on a replacement for libmemcached to hopefully cut down on the complexity... but then that's a migration and nobody wants to do that. Pinterest should drop me a line if they're interested in sponsoring work though :)

Post by aparsons » Sun, Jul 03, 2022, 6:24 AM UTC

mandeepj wrote: post #5

> the fleet serves up to ~180 million requests per second For comparison, Google serves about 63k queries per second. I hope there's not a typo in the above line in Pinterest's blog

Post by myrion » Sun, Jul 03, 2022, 7:12 AM UTC

Post by ddorian43 » Sun, Jul 03, 2022, 7:38 AM UTC

divyekapoor wrote: post #7

mandeepj wrote: post #5

> the fleet serves up to ~180 million requests per second For comparison, Google serves about 63k queries per second. I hope there's not a typo in the above line in Pinterest's blog

Google serves up way way more than 63k queries per second (several million requests/s is common for the critical services around search). Source: I worked there. However, your main point is valid (180 million/s seems way too high). I've started a thread internally to double check these numbers. Please wait for an update.