Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A multithreaded fork of Redis that is faster

docs.keydb.dev

Post reply on HN
Print view View original

21–30 of 164 posts

Re: A multithreaded fork of Redis that is faster

#21

Post by jdsully »

erulabs wrote: post #20

Any time these "much faster than Redis" databases come up, the sysadmin in me wonders how many people have had actual performance limitation issues with Redis. I've seen Redis servers handle hundreds of GB of traffic per hour. I've worked at companies where Aerospike and others are proposed as replacements for Redis because "they're faster" - and I point out the 98% idle CPUs on the Redis server, and the near-100%-us…

If the perf isn’t needed we also have Active Replication, subkey expiries, and S3 integration. Multithreading is how we got our start and the reason most people use us though.

Re: A multithreaded fork of Redis that is faster

#22

Post by nitwit005 »

moab wrote: post #6
tengbretson wrote: post #4

I've always considered the single-threaded nature of Redis to be one of its greatest features.

Can you elaborate on why? As someone who works on parallel/concurrent algorithms and data structures a single-threaded system design seems anathema to most research over the past decade (maybe excluding H-Store, which uses local single-threadedness in an interesting way).

You can run multiple instances of redis on the same machine, which gets you parallelism from multiple processes instead of multiple threads. There's some extra deployment effort in doing so, but it's not generally a big issue.

It also isn't exactly single threaded. It calls fork when it wants to persist data to disk, which is functionally similar to starting up a thread to do disk IO.

Re: A multithreaded fork of Redis that is faster

#23

Post by manigandham »

erulabs wrote: post #20

Any time these "much faster than Redis" databases come up, the sysadmin in me wonders how many people have had actual performance limitation issues with Redis. I've seen Redis servers handle hundreds of GB of traffic per hour. I've worked at companies where Aerospike and others are proposed as replacements for Redis because "they're faster" - and I point out the 98% idle CPUs on the Redis server, and the near-100%-us…

True, it's rarely just raw performance. KeyDB has other advantages like multi-threading and disk-based persistence (instead of being limited by RAM) that makes it better at utilizing your server resources and handling larger scales.

Re: A multithreaded fork of Redis that is faster

#24

Post by manigandham »

jadbox wrote: post #16

Since this has been around for awhile, why hasn't Redis adopted this strategy into core?

Redis is actually exploring multithreading for IO. Here is antirez/Salvatore talking about the performance results in the write path: https://twitter.com/antirez/status/1110973404226772995

Re: A multithreaded fork of Redis that is faster

#25

Post by zymhan »

jadbox wrote: post #16

Since this has been around for awhile, why hasn't Redis adopted this strategy into core?

From Antirez's (Redis Maintainer) blog:

> Another thing to note is that Redis is not Memcached, but, like memcached, is an in-memory system. To make multithreaded an in-memory system like memcached, with a very simple data model, makes a lot of sense. A multi-threaded on-disk store is mandatory. A multi-threaded complex in-memory system is in the middle where things become ugly: Redis clients are not isolated, and data structures are complex. A thread doing LPUSH need to serve other threads doing LPOP. There is less to gain, and a lot of complexity to add.

http://antirez.com/news/126

Re: A multithreaded fork of Redis that is faster

#26

Post by axaxs »

cheald wrote: post #12
femto113 wrote: post #7

Earlier quoted context omitted.

Only one thread can access the data at any given time, so it seems like most of the things you'd expect to be guaranteed by a single thread still are. I found this comment particularly interesting Unlike most databases the core data structure is the fastest part of the system. Most of the query time comes from parsing the REPL protocol and copying data to/from the network. I wonder if anyone in the Redis ecosphere ha…

The human-readable/writable protocol is one of my favorite things about Redis, tbh. I can see cases where a really optimized system could benefit from a binary protocol, but I suspect it'd be a loss for most people.

Why not just offer both?

Re: A multithreaded fork of Redis that is faster

#27

Post by tyingq »

femto113 wrote: post #7
tengbretson wrote: post #4

I've always considered the single-threaded nature of Redis to be one of its greatest features.

Only one thread can access the data at any given time, so it seems like most of the things you'd expect to be guaranteed by a single thread still are. I found this comment particularly interesting Unlike most databases the core data structure is the fastest part of the system. Most of the query time comes from parsing the REPL protocol and copying data to/from the network. I wonder if anyone in the Redis ecosphere ha…

The excerpted comment in a format mobile readers can see without left/right scrolling:

"Unlike most databases the core data structure is the fastest part of the system. Most of the query time comes from parsing the REPL protocol and copying data to/from the network."

Re: A multithreaded fork of Redis that is faster

#30

Post by not_a_cop75 »

kirstenbirgit wrote: post #28
not_a_cop75 wrote: post #14

I guess I can no longer use Redis. I mean what can the use case of normal Redis possibly be?

It works, it's simple, and it's battle tested.

Does anyone not read into the sarcasm? This is the same cliche format stating that x is better than y, in such a manner to supposedly shame people still using y.
Post reply on HN