Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance

redislabs.com

Post reply on HN
Print view View original

31–40 of 56 posts

Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance

#31

Post by dvirsky »

yaaminu wrote: post #30
immad wrote: post #18

Earlier quoted context omitted.

Redis won't function till it has loaded all data from disk to memory. So if your webapp doesn't have too much data then possibly. Also since Redis is memory backed you need more RAM then data. This can get very costly. Another annoyance is that Redis is single process single threaded so you really have to avoid running long running queries unless you do extensive manual sharding. (Disclaimer: it's been a few years si…

Redis can now be multi threaded in custom modules

Sort of - the model it supports is not really multi-threaded. The modules can spawn threads, and acquire a "GIL" when they want to touch actual Redis data - thus only one thread at a time actually "owns" the entire Redis instance.

This allows long running queries to do primitive cooperative multi-tasking, releasing the GIL and letting other queries have a chance; But there is no real parallel data access. You will only gain real parallelism if you have actual work to do that does not touch the data directly when a thread is not touching the GIL. There aren't many cases that this applies to - usually copying the data aside to do work on it is not worth the gain of parallelism.

Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance

#33

Post by cjhanks »

I guess I don't understand who requires less than 1ms latency on ACID writes in a system accessible only through socket interfaces. Even so - isn't this benchmark simply pushing the requirement of fast disk syncs onto a fast flash drive designed for DMA? I mean, okay.. I guess... did customers actually think the networking wasn't the latency culprit?

Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance

#34

Post by mayank »

kristoff_it wrote: post #20
mayank wrote: post #17

Earlier quoted context omitted.

You definitely could, with some pretty big caveats: * you can tolerate some data loss in case of system crashes/power failures/etc, because redis only flushes data to disk periodically [1] * your dataset can fit in RAM, since redis is an in-memory datastore [1] https://redis.io/topics/persistence

Using AOF you have better durability, you can read about it in the link you provided.

As far as I know, AOF is still not the default setting. It's important to point this out, or otherwise Redis will suffer the same fate as early MongoDB, which had a similar default persistence model which many users didn't fully understand.

Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance

#36

Post by lobster_johnson »

nosefouratyou wrote: post #32

I would think that something like hyperdex/warp[1] would be better for that use case. [1] http://hyperdex.org/ [2] https://www.cs.cornell.edu/people/egs/papers/hyperdex-sigcom... [3] http://rescrv.net/papers/warp-tech-report.pdf

Hyperdex is no longer maintained [1]. While the technology is impressive, the author seems to have lost interest, and is now working on something called Consus [2].

Hyperdex's problem all along was that the author — a very talented developer from what I can tell — seems more invested in his projects from the perspective academic research (he's at Cornell) than in delivering a practical, living open source project. He tried to form a company around Hyperdex (the transactional "Warp" add-on thing was commercial) even though nobody seemed to be using it; and he was the sole developer. Unfortunately, as interesting as Consus is, history seems to be repeating itself there.

But yeah, Hyperdex seemed to have real potential at one point. It was the only NoSQL K/V store (at the time) that had transactions.

[1] https://github.com/rescrv/HyperDex/issues/233

[2] https://github.com/rescrv/Consus

Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance

#37

Post by le-mark »

edem wrote: post #12
segmondy wrote: post #11

Very impressive, especially how it doesn't seem to matter much on the read/write ratio. I've only used redis to cache, do folks really use it as a DB?

I'd also like to know what else is it used for.

I used redis' lua scripting to implement a per user/account cache; I wanted to provide a memcached instance per account, but also enforce a limit on cache size so a single account couldn't cache GBs of data and detoriate the service for everyone. I used hashes to track items and their sizes per account, and simply calculated the available size of all live objects on insert.

Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance

#38

Post by misframer »

lobster_johnson wrote: post #36
nosefouratyou wrote: post #32

I would think that something like hyperdex/warp[1] would be better for that use case. [1] http://hyperdex.org/ [2] https://www.cs.cornell.edu/people/egs/papers/hyperdex-sigcom... [3] http://rescrv.net/papers/warp-tech-report.pdf

Hyperdex is no longer maintained [1]. While the technology is impressive, the author seems to have lost interest, and is now working on something called Consus [2]. Hyperdex's problem all along was that the author — a very talented developer from what I can tell — seems more invested in his projects from the perspective academic research (he's at Cornell) than in delivering a practical, living open source project. He…

> It was the only NoSQL K/V store (at the time) that had transactions.

What about FoundationDB?

Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance

#39

Post by overcast »

b34r wrote: post #35

Why wouldn't they say 500kops/s? Using a small number like 0.5 is just bad marketing.

Because anyone who actually understands any of that, knows it's an impressive number regardless. They aren't "marketing" this to the general public.

Re: Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance

#40

Post by mkj »

sgmansfield wrote: post #26

It's worth noting that they're running this benchmark on an absolutely enormous (and expensive) instance: https://aws.amazon.com/ec2/instance-types/x1/

Keep reading - AWS latency was too high so the benchmark was run on some bare Dell hardware.
Post reply on HN