NSQ – A realtime distributed messaging platform designed to operate at scale
1–10 of 57 posts
Re: NSQ – A realtime distributed messaging platform designed to operate at scale
#2Segment is probably the biggest NSQ user right now, and they're moving to Kafka - any employees want to weigh in? :)
Re: NSQ – A realtime distributed messaging platform designed to operate at scale
#3There is quite a bit of documentation on the design but I haven't seen anything more specific along the lines of a TLA+, Lean, etc specification.
There are plenty of projects like this and I'm curious how they go about creating specifications, checking their designs, etc.
Would the project benefit from a formal model or proofs? A colleague and I started a side project to provide specifications for core Openstack components but we're keeping our minds open to other projects as well.
Re: NSQ – A realtime distributed messaging platform designed to operate at scale
#4Segment is probably the biggest NSQ user right now, and they're moving to Kafka - any employees want to weigh in? :)
We run 1.5 million messages per minute through our NSQ framework and we're starting to run into architectural limitations i.r.t to the number of workers in each producer/consumer pool, and are now testing/benchmarking Kafka in our staging environment.
Re: NSQ – A realtime distributed messaging platform designed to operate at scale
#5Segment is probably the biggest NSQ user right now, and they're moving to Kafka - any employees want to weigh in? :)
We run 1.5 million messages per minute through our NSQ framework and we're starting to run into architectural limitations i.r.t to the number of workers in each producer/consumer pool, and are now testing/benchmarking Kafka in our staging environment.
I think the biggest question is, do you think its feasible to directly start with Kafka instead of NSQ or does Kafka just require a much stronger/larger team to operate than NSQ?
Re: NSQ – A realtime distributed messaging platform designed to operate at scale
#6what are the typical use cases for NSQ ?
Re: NSQ – A realtime distributed messaging platform designed to operate at scale
#7Earlier quoted context omitted.
We run 1.5 million messages per minute through our NSQ framework and we're starting to run into architectural limitations i.r.t to the number of workers in each producer/consumer pool, and are now testing/benchmarking Kafka in our staging environment.
I think the biggest question is, do you think its feasible to directly start with Kafka instead of NSQ or does Kafka just require a much stronger/larger team to operate than NSQ?
If you are starting off just use what you/the team is comfortable with. I mean, unless you think you will need capacity for over 25,000 messages per second.
Re: NSQ – A realtime distributed messaging platform designed to operate at scale
#8Earlier quoted context omitted.
We run 1.5 million messages per minute through our NSQ framework and we're starting to run into architectural limitations i.r.t to the number of workers in each producer/consumer pool, and are now testing/benchmarking Kafka in our staging environment.
I think the biggest question is, do you think its feasible to directly start with Kafka instead of NSQ or does Kafka just require a much stronger/larger team to operate than NSQ?
Kafka and NSQ have widely variant promises around things like durability, order, etc.
In most use cases you can get NSQ like behavior out of Kafka and the inverse isn't true. Kafka's performance and added gaurantees come at the expense of being harder to operate.
Re: NSQ – A realtime distributed messaging platform designed to operate at scale
#9what are the typical use cases for NSQ ?
probably better off looking at
Re: NSQ – A realtime distributed messaging platform designed to operate at scale
#10Earlier quoted context omitted.
I think the biggest question is, do you think its feasible to directly start with Kafka instead of NSQ or does Kafka just require a much stronger/larger team to operate than NSQ?
Kafka and NSQ have widely variant promises around things like durability, order, etc. In most use cases you can get NSQ like behavior out of Kafka and the inverse isn't true. Kafka's performance and added gaurantees come at the expense of being harder to operate.
Can someone summarize the promises? Specifically, would NSQ work well as an easier-to-operate, Kafka alternative, or are there low-throughput use cases it's just not suitable for?