Live data from Hacker News

Redis vs. KeyDB vs. Dragonfly vs. Skytable

news.ycombinator.com

31–40 of 56 posts

Re: Redis vs. KeyDB vs. Dragonfly vs. Skytable

#31
post #13

>Redis, which I'd like to call the "original" key/value store My old friend Memcached just came by and want to say Hello. He is still alive and kicking. Netflix has been treating him well.

It has cache in the name so people are less likely to use it inappropriately.

Re: Redis vs. KeyDB vs. Dragonfly vs. Skytable

#32
post #19

Earlier quoted context omitted.

Could you explain why?

Both are small projects by small teams (though I don't know how small Redis team is now). They have insane performance characteristics with rather small footprints and a wealth of functionality.

My experience is that small, dedicated teams of high-performing individuals are the only way to build tools like these. The level of architectural and design coherence starts to decrease at 5 people, and falls like a rock beyond 10-20.

The key problem is building a firewall so that such people can focus. The *organizational* problems there are hard. If you have a firewall around five people, you can't tell if:

- They're sitting at home drinking beer and watching cat videos

- Are developing the world's fastest database

Which leads to meetings, presentations, funding proposals, and so on. Which, in turn, leads to the need for large teams.

Re: Redis vs. KeyDB vs. Dragonfly vs. Skytable

#33
post #5

> Dragonfly expects you to have the latest hardware[6] and the latest kernel[7] on all your servers. I find this outrageous because not all servers run 5.10 and a majority of them are still using long-running branches on 4.x. I don't see anything wrong with that. Either your kv store is performing OK or can be scaled using the usual methods and it's not holding you back... or you actually critically depend on the fas…

As far as I can tell, "latest kernel" really means "has io_uring"...which is core to how Dragonfly works, so it seems reasonable to me.

Re: Redis vs. KeyDB vs. Dragonfly vs. Skytable

#34
I'm trying to get my workplace to accept Redis for simple caching but they're not open to the idea. Instead they're all in Hazelcast but real stingy about who gets to use it and for how much memory.

I just want to cache some objects and avoid hammer the DB on every request :´(

Re: Redis vs. KeyDB vs. Dragonfly vs. Skytable

#35

> Dragonfly expects you to have the latest hardware[6] and the latest kernel[7] on all your servers. I find this outrageous because not all servers run 5.10 I don’t understand why this is “outrageous”? There’s performance features and better hardware utilisation in newer kernels? If you want the performance offered by the applications built on top of these, then arguably you should be prepared to run the hardware and…

not if I can get equivalent (if not better) performance on my existing hardware

Re: Redis vs. KeyDB vs. Dragonfly vs. Skytable

#36
post #21

> I was NOT able to reproduce the 25X speed that Dragonfly claims The Dragonfly benchmark which showed 25x faster was run on c6gn.16xlarge which has 64 vCPU. The m5.4xlarge you used "only" has 16 vCPU. Since Dragonfly is achieving its performance by using multiple threads it's not weird that you weren't able to replicate its performance gain. In my opinion, when it comes to these types of multi-threaded benchmarks it…

If you start with "baseline, one-process performance", you're likely to mislead yourself, since no modern server has fewer than eight, and a single-thread benchmark can omit important locks. A modern baseline is probably two to four cores, even if you're running multiple services on one machine.

Re: Redis vs. KeyDB vs. Dragonfly vs. Skytable

#37

I'm trying to get my workplace to accept Redis for simple caching but they're not open to the idea. Instead they're all in Hazelcast but real stingy about who gets to use it and for how much memory. I just want to cache some objects and avoid hammer the DB on every request :´(

You could probably find a local caching library to do the same thing in-process (via mmap) and partially reduce workload that way. Hopefully, using a local library needs less managerial involvement than installing/using a service.

Re: Redis vs. KeyDB vs. Dragonfly vs. Skytable

#38
post #21

> I was NOT able to reproduce the 25X speed that Dragonfly claims The Dragonfly benchmark which showed 25x faster was run on c6gn.16xlarge which has 64 vCPU. The m5.4xlarge you used "only" has 16 vCPU. Since Dragonfly is achieving its performance by using multiple threads it's not weird that you weren't able to replicate its performance gain. In my opinion, when it comes to these types of multi-threaded benchmarks it…

agree with your article entirely. the dragonfly benchmark seems far too artificial to be true and used in practice (given the expensive hardware). my tests were on a relatively cheaper box, and Skytable topped all the other systems.

Re: Redis vs. KeyDB vs. Dragonfly vs. Skytable

#40
post #21

> I was NOT able to reproduce the 25X speed that Dragonfly claims The Dragonfly benchmark which showed 25x faster was run on c6gn.16xlarge which has 64 vCPU. The m5.4xlarge you used "only" has 16 vCPU. Since Dragonfly is achieving its performance by using multiple threads it's not weird that you weren't able to replicate its performance gain. In my opinion, when it comes to these types of multi-threaded benchmarks it…

If you start with "baseline, one-process performance", you're likely to mislead yourself, since no modern server has fewer than eight, and a single-thread benchmark can omit important locks. A modern baseline is probably two to four cores, even if you're running multiple services on one machine.

I think you misunderstand. You start with 1 then test 2, 4, 8… up to the largest core count you can. This shows you where the multi threading arch breaks down.
Post reply on HN