We are using NSQ at Hailo for sending and receiving billions of messages every day. It scales incredibly well and is extremely reliable. We'd be happy to speak more about this with anyone who's interested.
How do you handle reliability? I'm reading http://nsq.io/overview/features_and_guarantees.html and I'm curious how you've set up your architecture to handle node failures without losing messages, or if losing messages is acceptable in your use case. I'm also curious if you've had issues with duplicated delivery, where a message is delivered more than once, and how you've handled that case.
Hi. Good question. We actually have both use cases. In the case where we require reliability and guaranteed delivery we publish to multiple nodes and standard channels that persist messages in the case of subscribers disconnecting or overflow in what can't be buffered in memory. In the other case we just publish once and can use ephemeral channels. We scale clusters depending on usage but start with a minimum of 3 and use nsqlookupd to find topics in the cluster. In the case of things that can't handle duplicate deliveries we offload deduping to simple things memcached.