Postgres: A better message queue than Kafka?
1–10 of 47 posts
Re: Postgres: A better message queue than Kafka?
#2"Many haters want you to buy their message queue product" lol
Re: Postgres: A better message queue than Kafka?
#3Honestly that's comparing apples and oranges. Also, that title is clickbait. It feels like a PR article disguised as a tech article.
Re: Postgres: A better message queue than Kafka?
#4You can spew messages into sqs then have a lambda on sqs sending to pg.
Funny, we were going to use kafka too, but just sending to mysql worked just fine . One day that will change, obviously.
Re: Postgres: A better message queue than Kafka?
#5"The write path. We built a daemon that would select log entries that were older than two weeks, copy them into a file in S3, and delete the rows from the database"
Seriously... How can you ever consider saying "a rdbms is just fine as a kafka alternative" under those conditions ?
Re: Postgres: A better message queue than Kafka?
#6I don’t think this article made a compelling reason not to use Kafka. In fact it may have made the opposite point.
Wouldn’t it have been easier to just use Kafka?
Re: Postgres: A better message queue than Kafka?
#7You can spew messages into sqs then have a lambda on sqs sending to pg. Funny, we were going to use kafka too, but just sending to mysql worked just fine . One day that will change, obviously.
It works fine on the happy path, but if mysql goes down, you lose things. Having a distributed, resilient queue has availability benefits.
Re: Postgres: A better message queue than Kafka?
#8File this under "If a headline is asking a question, then the answer is NO." Honestly, I'm not even sure what point the author is trying to make besides "anecdotally and at low scale, unusual-for-the-purpose technology X solved problem Y." A near-infinite number of bad patterns can solve problems along happy paths and resolve plenty of edge cases to boot. 99.9% availability was a goal post here? There are systems where 7 9s is unacceptable. The author didn't even provide data backing the measurement goals.
Re: Postgres: A better message queue than Kafka?
#9A controversial, but a very pragmatic take.
Queues are great for semi-infinite scalability, but you rarely need it.
There's numerous subtle benefits to using db compared to regular message queues that are often overlooked.
Being able to delete, reorder, or edit specific messages can be a lifesaver when things go wrong.
Most common issue with queue-based systems is getting overwhelmed with messages. Either your consumers went down. Or your producers had a bug / executed too frequently.
Being able to recover with a simple SQL query is a blessing.
Re: Postgres: A better message queue than Kafka?
#10File this under "If a headline is asking a question, then the answer is NO." Honestly, I'm not even sure what point the author is trying to make besides "anecdotally and at low scale, unusual-for-the-purpose technology X solved problem Y." A near-infinite number of bad patterns can solve problems along happy paths and resolve plenty of edge cases to boot. 99.9% availability was a goal post here? There are systems whe…
What are those mythical systems where seven nines is unacceptable? What are the chances you're going to work on one of those?
Google Spanner for example is up to five nines. Are there a lot of systems that need to be three orders of magnitude more reliable than Google Ads?