Some background on the challenges around e-commerce search/product discovery/matching.
https://aws.amazon.com/blogs/industries/building-blocks-for-...
80k RPS is not just for a single database lookup, don’t be silly.
How Amazon uses chaos engineering to handle 80k requests per second
81–90 of 119 posts
Re: How Amazon uses chaos engineering to handle 80k requests per second
#82Earlier quoted context omitted.
Can anyone calculate the dollars-per-request revenue and profit? Would be interesting to see how much it costs Amazon to make money and make some connection to the request rate.
Back in The Day, rumor had it the detail page hosting/rendering would easily max out a single machine after only a handful of queries per second. I regrettably can't verify the truth of that, or if it still holds, but there is a LOT going on with a given /dp . "How many of those requests translate to actual purchases?" is the next question.
IIRC there was some memory leak issue in the custom rendering engine/language that was being used and for a time the solution was to reboot any VM that has taken more than $x requests. $x was a very small number
Re: How Amazon uses chaos engineering to handle 80k requests per second
#83M80K requests per seconds were doable in the 90's and you didn't need a super computer for it.
Re: How Amazon uses chaos engineering to handle 80k requests per second
#84M80K requests per seconds were doable in the 90's and you didn't need a super computer for it.
[deleted]
Re: How Amazon uses chaos engineering to handle 80k requests per second
#85I'm pretty unclear on the "how" here - but from what I can understand in the article the search resilience team injected properly tagged synthetic traffic into their system to do testing? That does seem like the kind of practice that could be part of healthy holistic approach - but the article elides a ton of details. I suppose the idea is that it promotes AWS services (with the idea of suggesting that this kind of r…
Oh... "Chaos systems"=="event-driven development". For those confused about an analogy to chaos theory... as far as i can tell, there isn't one. In physics a chaos system has small perturbations that lead to an instability. I would argue this is just a large perturbation that leads to certain instabilities. I would also classify this as "network systems fault tolerance" engineering. As a stand-alone article it is fin…
Who's drawing parallels with chaos theory? The origin is Netflix injecting "chaos" like taking servers down randomly. It's not tied to any scientific theory: https://en.m.wikipedia.org/wiki/Chaos_engineering
"Stress testing" might be more intuitive but that's already established as simply testing under high traffic
Re: How Amazon uses chaos engineering to handle 80k requests per second
#86Did someone mean to post this about ten years and accidentally set the publish date too far out?
[deleted]
Re: How Amazon uses chaos engineering to handle 80k requests per second
#87Your article shows you put in a lot of work, and was written well. But... A few thoughts... Service Owners usually tend to design and build their systems in a way which allows them to understand and know how the system will behave under the worst conditions. It is not difficult to test these conditions either. A bash script utilizing curl will suffice. If you have to use 'Chaos Engineering' to experiment your way int…
[deleted]
Re: How Amazon uses chaos engineering to handle 80k requests per second
#88This is just plain old testing, no?
[deleted]
Re: How Amazon uses chaos engineering to handle 80k requests per second
#89why would they need to do anything - including blogging - to handle 80kqps, that's ~one machine nowadays
[deleted]
Re: How Amazon uses chaos engineering to handle 80k requests per second
#90only 80k?
[deleted]