Is this different from the other module nginx push stream which offers websockets, long polling etc
[deleted]
Show HN: Nchan – a pub/sub server as an Nginx module
11–20 of 44 posts
Re: Show HN: Nchan – a pub/sub server as an Nginx module
#12This is a huge refactoring of an old project of mine -- the Nginx HTTP Push Module. I'm wondering if anyone here has used it. Most importantly, I want feedback on the documentation. Did I overcomplicate things? Does it need more examples? Does it need more live code? Is it too long? Too short? Etc.
I love the clean client API, e.g. the use of ETags with long-polling. In some ways it reminds me of our LiveResource protocol ( http://liveresource.org ). It's still a work in progress, but maybe we can consolidate ideas. It'd be cool if an LR client could be pointed at an Nchan resource.
[deleted]
Re: Show HN: Nchan – a pub/sub server as an Nginx module
#13Is this different from the other module nginx push stream which offers websockets, long polling etc
Yes, this is a different project, although they are related. Push Stream is a fork of my Nginx HTTP Push Module, so they both descend from the same original codebase. Push Stream uses a blocking concurrency model, whereas Nhan is completely non-blocking. In theory, this means Nchan should scale better. In practice, I haven't benchmarked it heavily enough yet to see a divergence.
Besides that, nchan and Push Stream offer different feature sets. For example, Nchan has horizontal scaling and persistence through redis, whereas push stream has customizable message transforms. There are other differences as well, but that would take a whole article to elaborate. I should probably write it soon.
Re: Show HN: Nchan – a pub/sub server as an Nginx module
#14This is a huge refactoring of an old project of mine -- the Nginx HTTP Push Module. I'm wondering if anyone here has used it. Most importantly, I want feedback on the documentation. Did I overcomplicate things? Does it need more examples? Does it need more live code? Is it too long? Too short? Etc.
I love the clean client API, e.g. the use of ETags with long-polling. In some ways it reminds me of our LiveResource protocol ( http://liveresource.org ). It's still a work in progress, but maybe we can consolidate ideas. It'd be cool if an LR client could be pointed at an Nchan resource.
Interesting, I'll read through this. Nchan code is quite modular, so it would be pretty easy to add some kind of negotiation like this. Let's talk more.
For reference, here's the old document I wrote for the long-polling protocol back in '09: https://pushmodule.slact.net/protocol.html
Re: Show HN: Nchan – a pub/sub server as an Nginx module
#15How does this compare to MQTT ?
I'm not familiar with MQTT, I've just glanced over the spec right now.
Nchan is basically a message broker with channels, optimized for message broadcast.
MQTT is a TCP-level protocol, whereas all the currently implemented subscriber clients for Nchan are HTTP-level (Longpoll, EventSource and Websocket, which begins with an HTTP request). MQTT subscribers and publishers could be implemented in nchan, but I haven't yet written any raw-TCP connection negotiation code, so I don't know how hard it would be. Aside from that, the subscriber code is very modular and adding another protocol like MQTT would be straightforward.
Re: Show HN: Nchan – a pub/sub server as an Nginx module
#16I'm using Nginx Push Stream with NodeJS for a high-scale chat system (Soon to be released, completely Open-Source). The dependency on Nginx always bothers me. How hard do you think it would be to do a system like yours but completely standalone ? Then we would be able to integrate it to other languages via plugins (NodeJS, Python, etc).
Re: Show HN: Nchan – a pub/sub server as an Nginx module
#17I'm using Nginx Push Stream with NodeJS for a high-scale chat system (Soon to be released, completely Open-Source). The dependency on Nginx always bothers me. How hard do you think it would be to do a system like yours but completely standalone ? Then we would be able to integrate it to other languages via plugins (NodeJS, Python, etc).
Nchan is about 12K lines of C, I'd say 2-5K of that is dealing with Nginx guts. To get rid of Nginx entirely, you'd need to add an event loop, forking and multiprocess management, config parsing and reloading, and shared memory allocation code. That's not a simple task, but it's certainly possible. The reason I built this on top of Nginx is precisely because I didn't want to handle those other things. Besides, nginx these days is a hulking scalable monster. What's the bother?
If you really don't want an nginx dependency, I'd say you're better off rolling your own pubsub server in Node.
Re: Show HN: Nchan – a pub/sub server as an Nginx module
#18This is wonderful. I hope this module gets added to openresty.
Re: Show HN: Nchan – a pub/sub server as an Nginx module
#19I'm using Nginx Push Stream with NodeJS for a high-scale chat system (Soon to be released, completely Open-Source). The dependency on Nginx always bothers me. How hard do you think it would be to do a system like yours but completely standalone ? Then we would be able to integrate it to other languages via plugins (NodeJS, Python, etc).
Nchan is about 12K lines of C, I'd say 2-5K of that is dealing with Nginx guts. To get rid of Nginx entirely, you'd need to add an event loop, forking and multiprocess management, config parsing and reloading, and shared memory allocation code. That's not a simple task, but it's certainly possible. The reason I built this on top of Nginx is precisely because I didn't want to handle those other things. Besides, nginx…
Okay, thanks for the information.
The problem with doing the pubsub in NodeJS entirely is the way NodeJS handle connections. Each is separated and is accompanied by a big overhead from NodeJS. What would be great is a way to interact with the pubsub server not by config but with an API. This would allows, for example, the execution of middlewares when a user publishes a message (to filter them or something else).
Right now, I'm using two websockets, one to Nginx PushStream only to receive messages and another to the NodeJS server to publish messages. The NodeJS server is used to parse the messages, format them, authenticate the user and apply the middlewares before publishing the message to Redis. Then, each NodeJS get the message from Redis and POST it to Nginx.
The problem here is that there is two sockets for each user and the complexity involved with the communication between Nginx and NodeJS. That's why a really performant standalone websocket engine with an extensive API would be awesome.
Re: Show HN: Nchan – a pub/sub server as an Nginx module
#20Earlier quoted context omitted.
Nchan is about 12K lines of C, I'd say 2-5K of that is dealing with Nginx guts. To get rid of Nginx entirely, you'd need to add an event loop, forking and multiprocess management, config parsing and reloading, and shared memory allocation code. That's not a simple task, but it's certainly possible. The reason I built this on top of Nginx is precisely because I didn't want to handle those other things. Besides, nginx…
Okay, thanks for the information. The problem with doing the pubsub in NodeJS entirely is the way NodeJS handle connections. Each is separated and is accompanied by a big overhead from NodeJS. What would be great is a way to interact with the pubsub server not by config but with an API. This would allows, for example, the execution of middlewares when a user publishes a message (to filter them or something else). Rig…
> the execution of middlewares when a user publishes a message (to filter them or something else)
You can do that with nchan: https://nchan.slact.net/details#authenticate-with-nchan_auth...
> Right now, I'm using two websockets, one to Nginx PushStream only to receive messages and another to the NodeJS server to publish messages.
You can also multiplex several websockets into one for the client.
I can't offer you a standalone server, but I can offer some pretty fancy features : )