41–46 of 46 posts
Anyone here use NSQ in production ( https://github.com/bitly/nsq ). If so what are your thoughts in comparison to the article?
(disclosure: i've contributed to nsq, after we began using it in production)
Anyone here use NSQ in production ( https://github.com/bitly/nsq ). If so what are your thoughts in comparison to the article?
we do; rather happy with it. do you have any specific questions about it? the article's approach is.. strange, so I'm not sure how I can meaningfully contrast from a high-level. (disclosure: i've contributed to nsq, after we began using it in production)
Also, are you missing any features in nsq?
Anyone here use NSQ in production ( https://github.com/bitly/nsq ). If so what are your thoughts in comparison to the article?
there are a few (growing) number of production installations of NSQ... our (bitly's) cluster spans a a few datacenters and hits peaks of 80k messages/second. I can answer any questions you have (one of the authors)
Also, how was performance using JSON data format compared with ProtoBuffers & MsgPack?
Earlier quoted context omitted.
And then you've made your own broker. At that point why not use an existing, more mature broker?
The zero in ZeroMQ is because brokerlessness is a strength in some aspects. A broker can be an SPOF, for instance. 0MQ makes it easy to do shared-nothing.
Besides SPOF is not an argument. All serious brokers have supported clustering and failover for quite a while, and they support persistency. I don't see a good reason why you may want to write a ZeroMQ-based broker instead of just using RabbitMQ.
Earlier quoted context omitted.
there are a few (growing) number of production installations of NSQ... our (bitly's) cluster spans a a few datacenters and hits peaks of 80k messages/second. I can answer any questions you have (one of the authors)
Any advice on dealing with de-duping? Also, how was performance using JSON data format compared with ProtoBuffers & MsgPack?
re: de-duping - there are lots of things to consider, I highly recommend reading through http://cs.brown.edu/courses/csci2270/archives/2012/papers/we..., it's a fantastic paper. At a high level the answer is idempotency.
What sort of use case are you thinking of? (context would help answer your de-dupe question)
Earlier quoted context omitted.
we do; rather happy with it. do you have any specific questions about it? the article's approach is.. strange, so I'm not sure how I can meaningfully contrast from a high-level. (disclosure: i've contributed to nsq, after we began using it in production)
Any gotchas moving from RabbitMQ to nsq? Also, are you missing any features in nsq?
to the contrary- deploying static binaries (instead of the erlang environment) simplified things nicely, in my opinion. to be fair, i (personally) don't have the requisite experience tuning BEAM which probably biases my preference.
while relatively high volume, our usage of RabbitMQ was straightforward and covered by the functionality offered in NSQ.
like most, we were using AMQP.. so the switch to NSQ's concise wire protocol (and the associated reduction in pkt/s) saved us a lot of pain given the highly-variable performance we see in the EC2 network.