https://archive.fo/4WMYg
Doing the lords work here I see
How Fast can A Single Instance of Redis be?
41–50 of 79 posts
Re: How Fast can A Single Instance of Redis be?
#42Anecdotally, my experience running Redis servers on AWS (both standalone EC2 instances and ElastiCache dedicated Redis instances) is that network latency is likely to become a barrier before anything else. We struggled with the same performance problems on both small-footprint small-payload Redis DBs and on large ones, and paying for the next tier of network connectivity (between our applications on EC2 and our Redis servers on EC2/ElastiCache) did the most to alleviate delays.
Re: How Fast can A Single Instance of Redis be?
#43Earlier quoted context omitted.
Maybe they should use this module that I can't read about to power their site. :P
Makes me feel a little better about my libµhttp/Objective-C/Objective-Smalltalk server (serving http://objective.st ), which held up just fine to the HN hug of death, running on the smallest digital ocean droplet. :-)
Those $5 droplets pack a surprising punch.
Re: How Fast can A Single Instance of Redis be?
#44I'd like to see the client code, see how it manages backpressure. (I'm probably being thick; the benchmark code is probably linked and I'm just not seeing it.) I recently maintained some nodejs & expressjs stuff. Neither the Redis clients (or the original developers of our stuff) have any concept of backpressure (throttling). In our case, HTTP request received would cause a Redis request. For whatever reason, express…
We've been running into the issue where our redis instance(s) randomly dies. I haven't been able to pin point the problem (nodejs+redis). Would love to hear your thoughts on some gotchas to look out for.
Re: How Fast can A Single Instance of Redis be?
#45Nice! I am also working on developing an extension for Redis and tried creating FlameGraphs as well but am not able to get them to work properly. Could you please share the commands you executed for the Flame Graphs? Would be greatly appreciated!
Already tried using '-fno-omit-frame-pointer' and '-O0'
# $CMD is a command starting a redis-server and creating traffic
perf record --freq=10000 --all-cpus -g -- $CMD
perf script --input=perf.data | ./stackcollapse-perf.pl > out.perf-folded
./flamegraph.pl out.perf-folded > perf-redis.svg
Re: How Fast can A Single Instance of Redis be?
#46Anecdotally, my experience running Redis servers on AWS (both standalone EC2 instances and ElastiCache dedicated Redis instances) is that network latency is likely to become a barrier before anything else. We struggled with the same performance problems on both small-footprint small-payload Redis DBs and on large ones, and paying for the next tier of network connectivity (between our applications on EC2 and our Redis…
This is very accurate. Seeing variations in GET / SET operations with ElastiCache up to 10000μs, which is way longer than the actual execution speed. After moving complex operations to transactions & even Lua scripts, performance is fairly acceptable again.
I assume the AWS network is pretty noisy.
Re: How Fast can A Single Instance of Redis be?
#47I'd like to see the client code, see how it manages backpressure. (I'm probably being thick; the benchmark code is probably linked and I'm just not seeing it.) I recently maintained some nodejs & expressjs stuff. Neither the Redis clients (or the original developers of our stuff) have any concept of backpressure (throttling). In our case, HTTP request received would cause a Redis request. For whatever reason, express…
> I'd like to see the client code, see how it manages backpressure.
That should (in theory) be handled on layer 7, not really on the network end (beyond regular TCP flow control), whereas this article is mostly about optimization on the network layer.
If you're fiddling with redis to the point where you're optimizing to use DPDK/OpenOnload/Exablaze/etc you've probably already exhausted the typical optimization paths (redis pipelining, tuning your queries, potentially implementing a stripped down client library implementing RESP, etc).
Re: How Fast can A Single Instance of Redis be?
#48"Redis is known as one of the fastest databases out there." Redis is not a database. Let's begin with that.
I very much beg to differ. Just because a database maps to memory and doesn’t commit to disk per transaction doesn’t make it less of a database.
So if not a database, what is it?
Re: How Fast can A Single Instance of Redis be?
#49Earlier quoted context omitted.
We are back up, had to move to a bigger server haha
I'm always curious what kind of setup people run that can not handle a thousand people connecting into HTTP.
Guessing it's a CMS without a configured deep-caching server in front of it, like Wordpress, That does some DB request(s) and a few addon modules on each article request.
That's not to say its' bad, but is probably seeing higher than normal traffic. I've been interested in static content generation again.
Re: How Fast can A Single Instance of Redis be?
#50Earlier quoted context omitted.
We are back up, had to move to a bigger server haha
I'm always curious what kind of setup people run that can not handle a thousand people connecting into HTTP.
I did a bit of sleuthing in the dev console.
The site is a static HTML site generated by Docusaurus [0]. That seems like it should be very fast and lightweight for serving tons of concurrent requests.
In the page's response header, it shows that the server is NGINX on Ubuntu. Ahhh, and "X-Powered-By: Express". I suspect that's the bottleneck, serving static assets via Node.js+Express, instead of directly with NGINX.
I also see in the document foot that it tries to load a script from http://localhost:35729/livereload.js. That could be a sign that the site is a development build, not optimized for production..?