Polling reliably at scale using DLQs
blog.smallcase.com
Polling reliably at scale using DLQs
1–5 of 5 posts
Re: Polling reliably at scale using DLQs
#2I would _not_ recommend using DLQs in ActiveMQ as a delivery delay mechanism. DLQs are meant to hold bad messages that can't be processed by your consumers. Conflating those with good messages most certainly sounds like a debugging nightmare. Imagine your database going down and thousands of messages heading to your DLQ... you'd want to replay those so you don't lose valuable data. Mixing that with stuff that needed to run into the future sounds like a true headache.
ActiveMQ is an amazing toolbox however. In fact, it contains the exact feature they want (and surprised they did not mention). Internally it is driven by a robust Quartz scheduler. You simple enable scheduling on your broker config (schedulerSupport="true") then all you would need to do is put a header on your messages AMQ_SCHEDULED_DELAY with the number of ms to delay.
Re: Polling reliably at scale using DLQs
#3I would _not_ recommend using DLQs in ActiveMQ as a delivery delay mechanism. DLQs are meant to hold bad messages that can't be processed by your consumers. Conflating those with good messages most certainly sounds like a debugging nightmare. Imagine your database going down and thousands of messages heading to your DLQ... you'd want to replay those so you don't lose valuable data. Mixing that with stuff that needed…
ActiveMQ is awesome. Very easy to set up, lots of features, fast enough for most workloads and extremely stable.
Re: Polling reliably at scale using DLQs
#4I would _not_ recommend using DLQs in ActiveMQ as a delivery delay mechanism. DLQs are meant to hold bad messages that can't be processed by your consumers. Conflating those with good messages most certainly sounds like a debugging nightmare. Imagine your database going down and thousands of messages heading to your DLQ... you'd want to replay those so you don't lose valuable data. Mixing that with stuff that needed…
Re: Polling reliably at scale using DLQs
#5I would _not_ recommend using DLQs in ActiveMQ as a delivery delay mechanism. DLQs are meant to hold bad messages that can't be processed by your consumers. Conflating those with good messages most certainly sounds like a debugging nightmare. Imagine your database going down and thousands of messages heading to your DLQ... you'd want to replay those so you don't lose valuable data. Mixing that with stuff that needed…
Unlike ActiveMQ, RabbitMQ doesn't automatically send undelivered messages to a default DLQ. So that doesn't apply in this use case.
The ActiveMQ header to schedule messages looks promising. Thanks for sharing, will check it out. :)