Live data from Hacker News

How Amazon uses chaos engineering to handle 80k requests per second

community.aws

11–20 of 119 posts

Re: How Amazon uses chaos engineering to handle 80k requests per second

#11
post #7

Earlier quoted context omitted.

Did the math, that's about 7 billion searches a day. That doesn't sound like a lot.

I mean, google does 8.5 billion per day. What would be a lot to you?

To be fair, a lot of various URL bars and input fields turn other activities into implicit Google queries.

Re: How Amazon uses chaos engineering to handle 80k requests per second

#12
post #2

I feel like Amazon search is one of the worst products I've ever used. It is a clusterf/ck of paid advertisements and obviously gamed results. I don't care how many requests/sec you get. If the results are horrible, what does it matter?

I'm sick of it being impossible to identify cheaply made products from high quality, durable products on Amazon. The rating system is flat out broken and there's an entire industry built around gaming those ratings.

I'm at the point that I rarely ever buy products on Amazon anymore. It's a total disgrace. On an ethical level, I wish I had the ability to say "I only want to be presented with results that weren't made in China or other slave societies".

Re: How Amazon uses chaos engineering to handle 80k requests per second

#13
I would love to know what software stack, hardware, and uplink connections in total they utilize to accomplish a real-world 80k request per second throughput. How many instances do you guys think Amazon runs for its primary e-commerce front-end stack? In total, and per region? Assuming they have a multi-region rollout.

If it's the real-deal, and not like people saying "Bun.js can serve 65k req/s+ (cough cough to localhost,)" that's impressive.

But I never see anyone talk about real-world numbers. Just synthetic poopoo.

I think I read the article correctly, but I think it only talks about how they introduce "chaos engineering," I didn't recall them talking about how they actually handle a volume of traffic like 80k req/s.

Re: How Amazon uses chaos engineering to handle 80k requests per second

#15
post #2

I feel like Amazon search is one of the worst products I've ever used. It is a clusterf/ck of paid advertisements and obviously gamed results. I don't care how many requests/sec you get. If the results are horrible, what does it matter?

Is there any alternative front end for Amazon? AI with some image-similarity smarts could do a much better job grouping/de-duping similar products.

Re: How Amazon uses chaos engineering to handle 80k requests per second

#16

I would love to know what software stack, hardware, and uplink connections in total they utilize to accomplish a real-world 80k request per second throughput. How many instances do you guys think Amazon runs for its primary e-commerce front-end stack? In total, and per region? Assuming they have a multi-region rollout. If it's the real-deal, and not like people saying "Bun.js can serve 65k req/s+ ( cough cough to loc…

> I would love to know what software stack, hardware, and uplink connections in total they utilize to accomplish a real-world 80k request per second throughput. How many instances do you guys think Amazon runs for its primary e-commerce front-end stack? In total, and per region? Assuming they have a multi-region rollout. > But I never see anyone talk about real-world numbers. Just synthetic poopoo.

The number probably changes all the time based on load. They'll never release these details because it's a competitive advantage to have the "how popular are they in $place at $time-of-day" data private.

When they do share numbers, it'll always be the most flattering and devoid of any context beyond the "wow" factor.

Re: How Amazon uses chaos engineering to handle 80k requests per second

#17
I'm pretty unclear on the "how" here - but from what I can understand in the article the search resilience team injected properly tagged synthetic traffic into their system to do testing? That does seem like the kind of practice that could be part of healthy holistic approach - but the article elides a ton of details. I suppose the idea is that it promotes AWS services (with the idea of suggesting that this kind of resiliency comes easier on their platform) - but this is a great example of how good writing strips things down to the barest details. I would love to take lessons from it but I think the details actually aren't here.

Re: How Amazon uses chaos engineering to handle 80k requests per second

#18
post #2

I feel like Amazon search is one of the worst products I've ever used. It is a clusterf/ck of paid advertisements and obviously gamed results. I don't care how many requests/sec you get. If the results are horrible, what does it matter?

Working as intended.

Re: How Amazon uses chaos engineering to handle 80k requests per second

#19
post #15
post #2

I feel like Amazon search is one of the worst products I've ever used. It is a clusterf/ck of paid advertisements and obviously gamed results. I don't care how many requests/sec you get. If the results are horrible, what does it matter?

Is there any alternative front end for Amazon? AI with some image-similarity smarts could do a much better job grouping/de-duping similar products.

This would be awesome... When I'm in a rabbit hole trying to find a product and I see three or four of roughly the same design I know not to bother any further unless I can find a reliable manufacturer website.

Re: How Amazon uses chaos engineering to handle 80k requests per second

#20

Surprised that 80K/second is called "massive" for Amazon.com's main search feature.

Google search is only 99k qps

I’m sure there’s massive variance and seasonality around that number
Post reply on HN