It's worth noting that they're running this benchmark on an absolutely enormous (and expensive) instance: https://aws.amazon.com/ec2/instance-types/x1/
[deleted]
Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance
41–50 of 56 posts
Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance
#42I don't like to admit but I still don't get it, could I use Redis as a main webapp backend?
[disclosure I'm from Redis Labs] - around 50% of our 7K+ paying customers use Redis as a database.
In general, you can setup an environment in which Redis is HA (replication + auto-failover) and persistent.
Redis Enterprise does it by default + backup and DR + some enhancements to the Redis storage layer (as mentioned in this blog. These enhancements together with the high end storage device by Dell-EMC allowed us to reach this throughout & latency. BTW, this test ran on a single node, you can scale it by just adding node(s) to the cluster.
Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance
#43Questions I would like to see answered: - How does oss redis compare to enterprise redis ? - What are the differences when running the same benchmarks with both oss redis and enterprise redis ? - what is the marginal utility of an additional cpu core/thread ? that is, what happens if I run those benchmarks on an AMD ThreadRipper ?
To add to what dvirsky said [and I'm also from Redis Labs]
- Redis Enterprise adds some enhancements to the Redis storage layer (details are in the blog).
- This benchmark only tested the Redis Enterprise. The idea was to show how fast Redis (Enterprise) can run on a single node with ACID transactions and still keep sub-millisecond latency. Note the hidden point - at the moment you cannot achieve sub-millisecond latency over the cloud (any cloud) storage, I mean persistent storage that is attached to an instance and not the local storage, which is ephemeral by design. So in-order to see how far we can go we decided to test it over Dell-EMC's VMAX that doesn't have these limitations
- In theory adding CPUs/cores can of course help, as you can add more shards to the Redis cluster and increase the parallelism when accessing the storage. That said, we haven't tested it over AMD ThreadRipper.
Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance
#44"almost always" atomicity means "not atomic". (if i guess correctly, there is no stage/commit phase for data mutations in the aof persistence, so i believe incremental changes are written to the AOF as the script runs so you can have partially-applied updates on restore if your redis process dies in the middle of executing a watch/multi/exec or lua script.) also, instances die all the time.
Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance
#45FWIW, I just get a blank page in FF with uBlock running. I believe it's the "naked-social-share" which various 3rd party extensions (Fanboy's) is blocking.
Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance
#46"almost always" atomicity means "not atomic". (if i guess correctly, there is no stage/commit phase for data mutations in the aof persistence, so i believe incremental changes are written to the AOF as the script runs so you can have partially-applied updates on restore if your redis process dies in the middle of executing a watch/multi/exec or lua script.) also, instances die all the time.
> also, instances die all the time
This where the durability argument has a problem as well.
At a first glance, this looks like an un-replicated system, which means that the loss of an instance is an availability nightmare.
The worrying quote from the article for me is this one
>> With Redis Enterprise, it is easy to create a master-only cluster and have all shards run on the same node
Another node has to be brought in and attached to the same network disk to restore access to that key range?
500k+ ops/sec is nothing to laugh about on a single node with 1:1 read-write ratios, however the fragility of this system is concerning.
Half a decade ago, I was working with row-level atomic ops in ZBase/Membase (set-with-cas[1]), which gets away with using replication instead of an ssd backing the durability of operations + an fsync - the 99% latencies were at 3-4ms, but the scalability and availability were baked in.
[1] - https://github.com/zbase/documentation/wiki/Data-Integrity
Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance
#47"almost always" atomicity means "not atomic". (if i guess correctly, there is no stage/commit phase for data mutations in the aof persistence, so i believe incremental changes are written to the AOF as the script runs so you can have partially-applied updates on restore if your redis process dies in the middle of executing a watch/multi/exec or lua script.) also, instances die all the time.
I'm not sure how the append only mode works but I tried to run a script (appendfsync directive set to always) which sets a new key in an endless loop and kill the server in the middle of the execution.
No changes are written to the AOF so I guess the operation is atomic.
Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance
#48Stopped reading after I saw the following statement - “All or nothing” property is almost always achieved, excluding cases like...
Almost always achieved? That actually means it is "never really achieved", fixed for you.
It is just sad that cheap marketing materials like this one are keep being pushed to the front page of NH.
Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance
#49Earlier quoted context omitted.
Hyperdex is no longer maintained [1]. While the technology is impressive, the author seems to have lost interest, and is now working on something called Consus [2]. Hyperdex's problem all along was that the author — a very talented developer from what I can tell — seems more invested in his projects from the perspective academic research (he's at Cornell) than in delivering a practical, living open source project. He…
> It was the only NoSQL K/V store (at the time) that had transactions. What about FoundationDB?
Not open source, though.
Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance
#50Questions I would like to see answered: - How does oss redis compare to enterprise redis ? - What are the differences when running the same benchmarks with both oss redis and enterprise redis ? - what is the marginal utility of an additional cpu core/thread ? that is, what happens if I run those benchmarks on an AMD ThreadRipper ?
To add to what dvirsky and Yiftach said, this video [1] from RedisConf17 gave me the best understanding of the storage layer of Enterprise Redis. Before that, it was hard for me to break through the marketing fluff. Amazing what it can do with NVMe and/or Optane.
[1] RedisConf17 - Building High Performance DB with Redis using Flash Memory - Cihan B. & Frank O. https://youtu.be/Tf8JRsE6w2U?list=PL83Wfqi-zYZF1MDKLr5djmLYU...