Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A multithreaded fork of Redis that is faster

docs.keydb.dev

Post reply on HN
Print view View original

111–120 of 164 posts

Re: A multithreaded fork of Redis that is faster

#111

Post by cobookman »

erulabs wrote: post #20

Any time these "much faster than Redis" databases come up, the sysadmin in me wonders how many people have had actual performance limitation issues with Redis. I've seen Redis servers handle hundreds of GB of traffic per hour. I've worked at companies where Aerospike and others are proposed as replacements for Redis because "they're faster" - and I point out the 98% idle CPUs on the Redis server, and the near-100%-us…

Happens more often than you think with larger tech companies. Especially if one has hot keys.

However yeah, 99% of all deployments could get by with SQLite/MySQL/Postgres performance amounts.

Re: A multithreaded fork of Redis that is faster

#112

Post by sriku »

wakatime wrote: post #75
erulabs wrote: post #20

Any time these "much faster than Redis" databases come up, the sysadmin in me wonders how many people have had actual performance limitation issues with Redis. I've seen Redis servers handle hundreds of GB of traffic per hour. I've worked at companies where Aerospike and others are proposed as replacements for Redis because "they're faster" - and I point out the 98% idle CPUs on the Redis server, and the near-100%-us…

We're using a Redis alternative called SSDB [1] in production because the entire data set doesn't need to fit in RAM. This saves us thousands of dollars per month in server costs. Like you say, latency gains is nominal. The main reason I see alternative databases used is one missing key feature that's custom to a specific use case. [1]: https://github.com/ideawu/ssdb

Did you test out ARDB [1] when you chose SSDB? I've been considering these for a while since I know I'm going to hit that choice point though that's not happened yet.

[1] https://github.com/yinqiwen/ardb

Re: A multithreaded fork of Redis that is faster

#113

Post by dang »

ca6f08f8f8 wrote: post #108
codegeek wrote: post #66

Earlier quoted context omitted.

I upvoted you to get you back up a bit but sarcasm is not an HN thing :) unless you complete it with more details.

You're definitely right that sarcasm isn't an HN thing. However I still get seriously worried about the personalities of the programmers here who seem unable to recognise sarcasm.

It's not about being unable. Readers here recognize sarcasm, but they also recognize what happens to a web forum where it is allowed to proliferate. That's why https://news.ycombinator.com/newsguidelines.html asks commenters not to be snarky on HN.

Lame internet humor tends to grow like kudzu, and the users who post it underestimate how lame it is, as scott_s pointed out well long ago: https://news.ycombinator.com/item?id=7609289. Sarcasm isn't quite the same but it's related.

Re: A multithreaded fork of Redis that is faster

#114

Post by fwip »

manigandham wrote: post #91
jrochkind1 wrote: post #88

Earlier quoted context omitted.

> ...it's rarely just raw performance. KeyDB has other advantages like multi-threading... What are the advantages of "multi-threading" other than for performance?

Taking advantage of all your cores. Redis is awkward at high capacity since most servers scale CPU with RAM and you'll end up with most cores doing nothing. You can manually run multiple instances on the same server but now they're separate databases with operations and sharding overhead. Multithreading IO also reduces the latency hit from disk-persistence and provides more concurrency and throughput, which is a grea…

That's just performance spelled differently.

Re: A multithreaded fork of Redis that is faster

#115

Post by drenvuk »

maxpert wrote: post #51

So this year on RedisConf Antirez demoed threaded version for Redis (only transport needs to be multi-threaded, core remains single threaded). Numbers were already amazing. I will pick the community version of Redis any day over forks.

I feel like this take is somewhat unfair. Antirez was against threading until jdsully proved that it worked in exactly the same way that you're saying, multi-threaded transport with a single threaded core. In addition Keydb allows users to use ssds in addition to ram only when that is a paid feature for Redis Lab's enterprise support.

jdsully spurred the implementation of two of the best changes that you might see in redis in the near future and you consider even using keydb pointless.

Yea, ok.

Re: A multithreaded fork of Redis that is faster

#116

Post by BubRoss »

tomjakubowski wrote: post #100
braindeath wrote: post #89

Earlier quoted context omitted.

That doesn’t mean multiple long running processes will share memory though unless they actually create a shared mapping, which I think the gp is referring to. In addition to tremendous memory waste this also kills cache performance.

I'd presume the persistence / I/O process would be pretty short lived: its job is to save a snapshot to disk so there's not much reason to let it live long.

How does that share data between multiple Redis processes?

Re: A multithreaded fork of Redis that is faster

#117

Post by ca6f08f8f8 »

dang wrote: post #113
ca6f08f8f8 wrote: post #108

Earlier quoted context omitted.

You're definitely right that sarcasm isn't an HN thing. However I still get seriously worried about the personalities of the programmers here who seem unable to recognise sarcasm.

It's not about being unable. Readers here recognize sarcasm, but they also recognize what happens to a web forum where it is allowed to proliferate. That's why https://news.ycombinator.com/newsguidelines.html asks commenters not to be snarky on HN. Lame internet humor tends to grow like kudzu, and the users who post it underestimate how lame it is, as scott_s pointed out well long ago: https://news.ycombinator.com/it…

Thanks dang. I still have one question though. If what you say is a sufficient explanation, then wouldn't we expect HN users to respond to sarcasm with "Please don't post sarcasm here", or a link to the community guidelines, rather than ?

I do however agree about the kudzu and that it is better to just keep it at zero. The purpose of my two posts here is to better understand the community within which I work, not to argue for HN policy change.

Re: A multithreaded fork of Redis that is faster

#118

Post by haimez »

bnegreve wrote: post #68
BubRoss wrote: post #38

Earlier quoted context omitted.

If redis isn't multi-threaded the more cores and threads you have, the less CPU usage the machine would show, even if Redis is being stressed.

Except if you monitor per CPU usage, as you should.

If you use cloudwatch, the “should” part is tricky. “Should” you be able to and “should” you do it if metrics were reasonably priced based on their basic required storage / computational footprint? Yes.

However, “should” you in a world where you only have cloudwatch because you don’t want to roll your own monitoring or bring in a third party vendor? Probably not. It’s not a good thing, but it’s a reality. 1/n approximation is your go to here.

Re: A multithreaded fork of Redis that is faster

#119

Post by wakatime »

sriku wrote: post #112
wakatime wrote: post #75

Earlier quoted context omitted.

We're using a Redis alternative called SSDB [1] in production because the entire data set doesn't need to fit in RAM. This saves us thousands of dollars per month in server costs. Like you say, latency gains is nominal. The main reason I see alternative databases used is one missing key feature that's custom to a specific use case. [1]: https://github.com/ideawu/ssdb

Did you test out ARDB [1] when you chose SSDB? I've been considering these for a while since I know I'm going to hit that choice point though that's not happened yet. [1] https://github.com/yinqiwen/ardb

Looked at it, but setup for SSDB was easier and already was using SSDB in production. Hesitant to change something that's working so well. Do have periodic connection issues with SSDB but so infrequent it's easy to work around client-side. Using the python redis [1] client with Flask.

[1] https://pypi.org/project/redis/

Re: A multithreaded fork of Redis that is faster

#120

Post by [deleted] »

erulabs wrote: post #20

Any time these "much faster than Redis" databases come up, the sysadmin in me wonders how many people have had actual performance limitation issues with Redis. I've seen Redis servers handle hundreds of GB of traffic per hour. I've worked at companies where Aerospike and others are proposed as replacements for Redis because "they're faster" - and I point out the 98% idle CPUs on the Redis server, and the near-100%-us…

[deleted]
Post reply on HN