Redis vs. Kafka vs. RabbitMQ
blog.devgenius.io
Redis vs. Kafka vs. RabbitMQ
1–10 of 44 posts
Re: Redis vs. Kafka vs. RabbitMQ
#2It's worth noting that there are several projects built on top of Redis pub-sub that add more functionality, such as BullMQ: https://github.com/taskforcesh/bullmq
Re: Redis vs. Kafka vs. RabbitMQ
#3Expect performance issues when running RabbitMQ with persistence? What kind of issues?
Also the comment about Redis not being persistent, Redis has lots of persistence options that allow you to choose trade offs between durability and performance, with the most strict setting using an append only log of every query.
Decent 1000 mile view of these solutions, but some more depth would have been nice.
Re: Redis vs. Kafka vs. RabbitMQ
#4It's worth noting that there are several projects built on top of Redis pub-sub that add more functionality, such as BullMQ: https://github.com/taskforcesh/bullmq
Bull is great ! It's only NodeJS but I used it at work and it has been super easy to setup, use, and maintain.
Re: Redis vs. Kafka vs. RabbitMQ
#5Is anyone else considering using DyanmoDB for event sourcing instead of Kafka? We have a project at work where we store events into an event table. We then had a stream that invokes a distributor lambda. The distributor lambda lookups subscribers to a given event type. For each subscriber for a given event, we place a copy of the event to a SQS queue for the event subscriber. Each subscriber lambda can process events from its own queue. If processing fails, each subscriber will retry processing a configured number of times before moving the failed event to a dead letter queue.
This approach has been working well for us without the need of Kafka with our serverless apps. I was curious if anyone is doing something similar.
Re: Redis vs. Kafka vs. RabbitMQ
#6Just a fun question: is there a comparison that compares how well a simple PostgreSQL-based queue[0] fares against these specialized solutions?
[0] https://blog.crunchydata.com/blog/message-queuing-using-nati...
Re: Redis vs. Kafka vs. RabbitMQ
#7I treat Redis a bit like I treat PostgreSQL.
It's what I use when I don't know what I should use.
(In fact these days I might even be tempted to start with a simple PostgreSQL based queue and only swap to Redis later if it becomes clear that's what's needed).
I guess a better approach might be to carefully analyse requirements up front but if those requirements aren't known at the time you start the project it's useful just to get you going.
Re: Redis vs. Kafka vs. RabbitMQ
#8I dont understand why almost nobody wants to understand the difference between message brokers and a distributed log and its implications.
Re: Redis vs. Kafka vs. RabbitMQ
#9Expect performance issues when running RabbitMQ with persistence? What kind of issues? Also the comment about Redis not being persistent, Redis has lots of persistence options that allow you to choose trade offs between durability and performance, with the most strict setting using an append only log of every query. Decent 1000 mile view of these solutions, but some more depth would have been nice.
[deleted]
Re: Redis vs. Kafka vs. RabbitMQ
#10Expect performance issues when running RabbitMQ with persistence? What kind of issues? Also the comment about Redis not being persistent, Redis has lots of persistence options that allow you to choose trade offs between durability and performance, with the most strict setting using an append only log of every query. Decent 1000 mile view of these solutions, but some more depth would have been nice.
There issues if you run with mirroring. It's advised that you use quorum queues here:
https://www.cloudamqp.com/blog/reasons-you-should-switch-to-...
However many people using Rabbit are using it with Celery which doesn't work with quorum queues - so people learn to expect performance issues.