I wish Pinterest were less effective at making google image search useless, not more efficient...
Between Pinterest and now Britannica filling up normal google search, I've started using Bing unironically.
Improving distributed caching performance and efficiency at Pinterest
11–20 of 25 posts
Re: Improving distributed caching performance and efficiency at Pinterest
#12> Today, Pinterest's memcached fleet spans over 5000 EC2 instances across a variety of instance types optimized along compute, memory, and storage dimensions. Collectively, the fleet serves up to ~180 million requests per second and ~220 GB/s of network throughput over a ~460 TB active in-memory and on-disk dataset, partitioned among ~70 distinct clusters.
Wow.
Assuming $0.09 per GB egress on EC2, that's $51,321,600/mo. Of course, they must be on some Enterprise plan of some sort, but how much discount must they get to make it affordable?
By comparison, 180m requests per second on an "egress-free" serverless compute like Workers would cost $77,760,000/mo (assuming 6m per $1) or $233,280,000/mo (2m per $1).
Cloud is wild.
Re: Improving distributed caching performance and efficiency at Pinterest
#13No post body was provided.
Re: Improving distributed caching performance and efficiency at Pinterest
#14> Today, Pinterest's memcached fleet spans over 5000 EC2 instances across a variety of instance types optimized along compute, memory, and storage dimensions. Collectively, the fleet serves up to ~180 million requests per second and ~220 GB/s of network throughput over a ~460 TB active in-memory and on-disk dataset, partitioned among ~70 distinct clusters. Wow. Assuming $0.09 per GB egress on EC2, that's $51,321,600/…
I wouldn't expect memcached servers to have any egress. Still I'd bet they're paying a lot for whatever their egress actually is.
Re: Improving distributed caching performance and efficiency at Pinterest
#15I wish Pinterest were less effective at making google image search useless, not more efficient...
Just use the two following uBlock Origin / Adguard rules, the first one for text results, the second one for images:
google.*##.g:has(a[href*=".pinterest."])
google.*##a[href*=".pinterest."]:upward(1)
Re: Improving distributed caching performance and efficiency at Pinterest
#16I wish Pinterest were less effective at making google image search useless, not more efficient...
Just use the two following uBlock Origin / Adguard rules, the first one for text results, the second one for images: google.*##.g:has(a[href*=".pinterest."]) google.*##a[href*=".pinterest."]:upward(1) You can build custom rules for other websites and other search engines at https://letsblock.it/filters/search-results
You can also use the uBlacklist extension which supports many websites and can block arbitrary domains.
Re: Improving distributed caching performance and efficiency at Pinterest
#17> the fleet serves up to ~180 million requests per second For comparison, Google serves about 63k queries per second. I hope there's not a typo in the above line in Pinterest's blog
Their numbers seem reasonable.
As an anecdote, I've worked at a scale up that handled ~20M HTTP(S) req/min on their service mesh.
The rule of thumb was that our Redis cache layer would have 10x the number of HTTP request, so 200M req/min.
Note that I'm using minutes, not seconds on the units. But it wouldn't surprise me that Pinterest handles 60x the load that we had back then.
Re: Improving distributed caching performance and efficiency at Pinterest
#18> the fleet serves up to ~180 million requests per second For comparison, Google serves about 63k queries per second. I hope there's not a typo in the above line in Pinterest's blog
Google serves up way way more than 63k queries per second (several million requests/s is common for the critical services around search). Source: I worked there. However, your main point is valid (180 million/s seems way too high). I've started a thread internally to double check these numbers. Please wait for an update.
> Google serves up way way more than 63k queries per second (several million requests/s is common for the critical services around search). Source: I worked there.
All right. But, here's my source - https://www.google.com/search?q=how+many+requests+does+googl... . Maybe, it's flawed
Re: Improving distributed caching performance and efficiency at Pinterest
#19> Today, Pinterest's memcached fleet spans over 5000 EC2 instances across a variety of instance types optimized along compute, memory, and storage dimensions. Collectively, the fleet serves up to ~180 million requests per second and ~220 GB/s of network throughput over a ~460 TB active in-memory and on-disk dataset, partitioned among ~70 distinct clusters. Wow. Assuming $0.09 per GB egress on EC2, that's $51,321,600/…
It doesnt really matter how much they pay for cloud as long as its less than what they get from Google for the image spam/pollution.
Re: Improving distributed caching performance and efficiency at Pinterest
#20Curious on why self hosted MC vs Elasticache MC?