Surprised that 80K/second is called "massive" for Amazon.com's main search feature.
Google search is only 99k qps
Bigtable does 6B QPS though…
https://cloud.google.com/blog/products/databases/youtube-run...
How Amazon uses chaos engineering to handle 80k requests per second
41–50 of 119 posts
Re: How Amazon uses chaos engineering to handle 80k requests per second
#42Surprised that 80K/second is called "massive" for Amazon.com's main search feature.
That's pretty crazy. At a big social media company, a service I ran got 300K+ requests per second directly from end users.
Not all requests per second are made the same.
For example, it is just as true for this title to have said "How Amazon uses ... to load 1.6 MM requests per second, from just the search page."
Each search page load, is 1 request to the search backend, but 20x request fanout to the product's key-value store to render the images and titles, etc.
Re: How Amazon uses chaos engineering to handle 80k requests per second
#43I would love to know what software stack, hardware, and uplink connections in total they utilize to accomplish a real-world 80k request per second throughput. How many instances do you guys think Amazon runs for its primary e-commerce front-end stack? In total, and per region? Assuming they have a multi-region rollout. If it's the real-deal, and not like people saying "Bun.js can serve 65k req/s+ ( cough cough to loc…
I used to work on a system that did about 55k/sec at peak. The service was internal it was only handling grpc calls which were coming from inside our VPC and it was written in Go. It's main job was was reading and writing to an SQL db that was sharded across 3 or 4 of the biggest instances AWS offered at the time (2017ish).
Everything was Dockerized and I think we were using Docker Swarm for container orchestration. I don't remember the specs for each box, but we had auto scaling set up so at peak we'd hit a little over 200 containers.
Looking back now, I'm sure we could have gotten much better performance out of that service, but the team was young and inexperienced and throwing money at the problem was an easier solution.
Re: How Amazon uses chaos engineering to handle 80k requests per second
#44I would love to know what software stack, hardware, and uplink connections in total they utilize to accomplish a real-world 80k request per second throughput. How many instances do you guys think Amazon runs for its primary e-commerce front-end stack? In total, and per region? Assuming they have a multi-region rollout. If it's the real-deal, and not like people saying "Bun.js can serve 65k req/s+ ( cough cough to loc…
> If it's the real-deal, and not like people saying "Bun.js can serve 65k req/s+ (cough cough to localhost,)" that's impressive.
Not all req/s are made the same.
Amazon search is made of 100s of services, and Amazon's search page loads 20 products per page, that means 80k search req/s translates to 1.6 MM product API req/s for example.
FWIW a search request at Amazon hits roughly 100 unique search clusters (think of this as ElasticSearch clusters - but its not ES), with different product groupings in each cluster. Each cluster is made up of 1000s of nodes running Lucene (think similar to ES shards). This is just for the "match set", i.e. which products to return.
Then there are services to re-sort those matched products based on popularity, likelihood of purchase, etc. Think giant ML models. Then there are product lookups. Before all of this, there is Query analysis to simplify/improve the query (think giant ML models) to classify "Apple" into electronics vs groceries based on the other keywords and your current context.
Meanwhile, Bun.js is talking about 65k "hello world" type req/s. The compute per req is magnitudes different.
Re: How Amazon uses chaos engineering to handle 80k requests per second
#45why would they need to do anything - including blogging - to handle 80kqps, that's ~one machine nowadays
Re: How Amazon uses chaos engineering to handle 80k requests per second
#46only 80k?
Can anyone calculate the dollars-per-request revenue and profit? Would be interesting to see how much it costs Amazon to make money and make some connection to the request rate.
Re: How Amazon uses chaos engineering to handle 80k requests per second
#47why would they need to do anything - including blogging - to handle 80kqps, that's ~one machine nowadays
Your average enterprise Java backend cannot handle that.
Re: How Amazon uses chaos engineering to handle 80k requests per second
#48I feel like Amazon search is one of the worst products I've ever used. It is a clusterf/ck of paid advertisements and obviously gamed results. I don't care how many requests/sec you get. If the results are horrible, what does it matter?
I'm sick of it being impossible to identify cheaply made products from high quality, durable products on Amazon. The rating system is flat out broken and there's an entire industry built around gaming those ratings. I'm at the point that I rarely ever buy products on Amazon anymore. It's a total disgrace. On an ethical level, I wish I had the ability to say "I only want to be presented with results that weren't made…
I understand how you feel. It can be frustrating.
Contrary to popular belief Amazon actually does put energy into making sure products are responsibly sourced. Products are de-listed if they’re found to come from unethical sources.
To take that even further take a look at Climate Pledge Friendly. Those are products with (at least one) third party certification. These certifications don’t just further climate goals. Social responsibility is also considered. Including worker conditions and product durability. You can filter search results by this attribute. Admittedly it can be hard to filter for specific certifications.
Re: How Amazon uses chaos engineering to handle 80k requests per second
#49Re: How Amazon uses chaos engineering to handle 80k requests per second
#50I would love to know what software stack, hardware, and uplink connections in total they utilize to accomplish a real-world 80k request per second throughput. How many instances do you guys think Amazon runs for its primary e-commerce front-end stack? In total, and per region? Assuming they have a multi-region rollout. If it's the real-deal, and not like people saying "Bun.js can serve 65k req/s+ ( cough cough to loc…
Amazon doesn't really have a "primary e-commerce front-end stack" in any concrete sense. They have hundreds/thousands of teams that deploy bits and pieces to a massive pipeline that ultimately makes up what you see on Amazon.com, but each team can have their own infrastructure backing things. Some teams might run everything off a dozen low-end EC2 instances while another sibling team has 3k+ instances; it's really al…
Thanks for the neat insight!