Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Nchan – a pub/sub server as an Nginx module

nchan.slact.net

Post reply on HN
Print view View original

31–40 of 44 posts

Re: Show HN: Nchan – a pub/sub server as an Nginx module

#31

Post by ende42 »

slact wrote: post #2

This is a huge refactoring of an old project of mine -- the Nginx HTTP Push Module. I'm wondering if anyone here has used it. Most importantly, I want feedback on the documentation. Did I overcomplicate things? Does it need more examples? Does it need more live code? Is it too long? Too short? Etc.

Thanks for your work!

Actually we (laut.fm) are using the HTTP Push Module for our public API for a live stream of tracks which get played on our ~ 1500 icecast stations. We offer 3 formats: http://api.laut.fm/song_change.stream.json for a line separated JSON HTTP stream, ws://api.laut.fm/song_change.ws.json for the same as websocket endpoint and http://api.laut.fm/song_change.chunk.json for the last x songs. It's not really high volume, just 6 to 7 per second. But it runs basically unattended for years now and I'm pretty happy with it.

Is there any reason to update to the new one (other than new features; admittedly I haven't really looked into NCHAN)?

Re: Show HN: Nchan – a pub/sub server as an Nginx module

#32

Post by slact »

ende42 wrote: post #31
slact wrote: post #2

This is a huge refactoring of an old project of mine -- the Nginx HTTP Push Module. I'm wondering if anyone here has used it. Most importantly, I want feedback on the documentation. Did I overcomplicate things? Does it need more examples? Does it need more live code? Is it too long? Too short? Etc.

Thanks for your work! Actually we (laut.fm) are using the HTTP Push Module for our public API for a live stream of tracks which get played on our ~ 1500 icecast stations. We offer 3 formats: http://api.laut.fm/song_change.stream.json for a line separated JSON HTTP stream, ws://api.laut.fm/song_change.ws.json for the same as websocket endpoint and http://api.laut.fm/song_change.chunk.json for the last x songs. It's no…

Well, if it ain't broke, as they say...

here's a page on the differences between Nchan and the Push Module: https://nchan.slact.net/upgrade . One important thing I forgot to add is that the Push Module suffered from memory fragmentation under high load, and with a fixed-size shared-memory chunk that could mean running out of usable shared memory for a long-running nginx process. If you're not experiencing that, and you don't need to scale up, or the new features don't appeal to you, don't upgrade -- certainly not yet.

Maybe in a month or two when nchan makes its way into the nginx-extras debian package (replacing the push module), then consider upgrading.

Re: Show HN: Nchan – a pub/sub server as an Nginx module

#33

Post by sams99 »

This is super interesting, I have a very similar project that we use at Discourse https://github.com/SamSaffron/message_bus clearly being an NGINX module nchan is going to be a LOT faster

That said there are few little pointers that would make integrating MessageBus with nchan a bit challenging

- We have a concept of "reliable pub sub" https://github.com/SamSaffron/message_bus/blob/master/lib/me... this means that if you shut your laptop and then open it stuff just magically catches up with no loss of messages or ordering. Does the redis store you have do something similar (I can not see backing lists in the redis store implementation in store.c)

- We implement per-message security, meaning you can publish to a channel/group or channel/user and only those users get the message, I see you can do auth upfront but can per-message security seems not doable.

- We don't bother with per-channel URLS and instead just have a single endpoint you multi-subscribe using POST params, we often subscribe to 10 channels on a request, the multiplexing seems a bit limited

- When we subscribe to a channel due to the "reliable" nature of the store we are able to tell the server what the last id is we subscribed to and catch up from there

I definitely see us moving the moving our message bus server piece closer to the web server like nchan has, its by far the most scalable way, but its tough keeping feature parity.

Re: Show HN: Nchan – a pub/sub server as an Nginx module

#34

Post by slact »

sams99 wrote: post #33

This is super interesting, I have a very similar project that we use at Discourse https://github.com/SamSaffron/message_bus clearly being an NGINX module nchan is going to be a LOT faster That said there are few little pointers that would make integrating MessageBus with nchan a bit challenging - We have a concept of "reliable pub sub" https://github.com/SamSaffron/message_bus/blob/master/lib/me... this means that if…

Nice project, and a curious featureset overlap, too...

> We have a concept of "reliable pub sub" [...] Does the redis store you have do something similar (I can not see backing lists in the redis store implementation in store.c)

Dig a little deeper... https://github.com/slact/nchan/blob/master/src/store/redis/s... I used lua scripts for all the fancy redis logic, that's where you'll find the backing list accesses. Messages are stored as hashes, referenced by id in lists, along with some other channel metadata. So a longpoll or EventSource client knows its last message id, and can request the next avaliable message as long as said message has not yet expired. Websocket clients don't have this information, and I'm not yet sure how to relay it with each message while remaining content-agnostic. Basically, regardless of the protocol, you can send a If-Modified-Since + If-None-Match or Last-Event-ID headers and it will resume from that position in the message queue for the given channel.

> I see you can do auth upfront but can per-message security seems not doable

Per-message access will definitely not be implemented. You could, however, do this client-side by, say, encrypting the messages and sharing keys with authorized subscribers. That's kind of roundabout though.

> We don't bother with per-channel URLS [...], we often subscribe to 10 channels on a request, the multiplexing seems a bit limited

The main use-case I had in mind for multiplexing is that of a single channel per user, and some shared broadcast channel. It's currently limited to 4 max because I wanted to get this code out the door. Unlimited multiplexing will be supported in the future, and you could trivially rebuild the module to support up to 16 right now (At the cost of some memory per message per subscriber per channel overhead).

> When we subscribe to a channel due to the "reliable" nature of the store we are able to tell the server what the last id is we subscribed to and catch up from there

Yep, Nchan does that too. (except for Websocket, and hopefully I'll find a workaround)

How does that work for feature parity?

Re: Show HN: Nchan – a pub/sub server as an Nginx module

#36

Post by sams99 »

slact wrote: post #34
sams99 wrote: post #33

This is super interesting, I have a very similar project that we use at Discourse https://github.com/SamSaffron/message_bus clearly being an NGINX module nchan is going to be a LOT faster That said there are few little pointers that would make integrating MessageBus with nchan a bit challenging - We have a concept of "reliable pub sub" https://github.com/SamSaffron/message_bus/blob/master/lib/me... this means that if…

Nice project, and a curious featureset overlap, too... > We have a concept of "reliable pub sub" [...] Does the redis store you have do something similar (I can not see backing lists in the redis store implementation in store.c) Dig a little deeper... https://github.com/slact/nchan/blob/master/src/store/redis/s... I used lua scripts for all the fancy redis logic, that's where you'll find the backing list accesses. Me…

nice! I initially stayed away from lua due so I could support earlier redises, but these days I would totally take that dependency on to simplify the code.

regarding that implementation, one diff I am spotting is that I also have a concept of global message id / global channel, this allows us to subscribe to a single spot and distribute from there (which keeps thread counts down and is particularly useful for server -> server comms.)

A lot of parity with our projects :)

Regarding web sockets, I decided against supporting them (initial implementations did) the issue is that web sockets are 100% flaky on HTTP and just add tons of unneeded complexity, in advent of HTTP/2 in NGINX they would be a net loss imo cause you would be wasting a connection.

Re: Show HN: Nchan – a pub/sub server as an Nginx module

#37

Post by slact »

pablomolnar wrote: post #35

Great project! Is there any concept of ack/nack a message? Let's say the subscriber get a message and failed before processing it. Does the message go back to the queue?

No, ack/nack would need to be implemented in the client or the application.

However, messages do not disappear from the queue when received. All subscriber requests are idempotent, and can be repeated so long as the message queue is storing the message (which is a configurable parameter). So if a subscriber failed before processing it, it's free to request the same message again.

Re: Show HN: Nchan – a pub/sub server as an Nginx module

#38

Post by slact »

sams99 wrote: post #36
slact wrote: post #34

Earlier quoted context omitted.

Nice project, and a curious featureset overlap, too... > We have a concept of "reliable pub sub" [...] Does the redis store you have do something similar (I can not see backing lists in the redis store implementation in store.c) Dig a little deeper... https://github.com/slact/nchan/blob/master/src/store/redis/s... I used lua scripts for all the fancy redis logic, that's where you'll find the backing list accesses. Me…

nice! I initially stayed away from lua due so I could support earlier redises, but these days I would totally take that dependency on to simplify the code. regarding that implementation, one diff I am spotting is that I also have a concept of global message id / global channel, this allows us to subscribe to a single spot and distribute from there (which keeps thread counts down and is particularly useful for server…

> global message id / global channel Yeah, I don't have global ids. every message is bound to the channel it was published to. But I do understand that for your use case, (user + arbitrary list of groups), you'd need a good deal more than 4 channel multiplexing. It's a pretty strong use case and I'll see what I can do in the next month.

Of course, I can be incentivized to work faster with a generous donation :)

> Regarding web sockets [...]

All the cool kids were talking about it, so I thought I might as well support them, too.

Re: Show HN: Nchan – a pub/sub server as an Nginx module

#39

Post by ende42 »

slact wrote: post #32
ende42 wrote: post #31

Earlier quoted context omitted.

Thanks for your work! Actually we (laut.fm) are using the HTTP Push Module for our public API for a live stream of tracks which get played on our ~ 1500 icecast stations. We offer 3 formats: http://api.laut.fm/song_change.stream.json for a line separated JSON HTTP stream, ws://api.laut.fm/song_change.ws.json for the same as websocket endpoint and http://api.laut.fm/song_change.chunk.json for the last x songs. It's no…

Well, if it ain't broke, as they say... here's a page on the differences between Nchan and the Push Module: https://nchan.slact.net/upgrade . One important thing I forgot to add is that the Push Module suffered from memory fragmentation under high load, and with a fixed-size shared-memory chunk that could mean running out of usable shared memory for a long-running nginx process. If you're not experiencing that, and y…

Thanks for the upgrade link. I missed that. Great that NCHAN is (almost; besides directive prefix) configuration compatible!

The "Subscribers" section of "Push Module" should include websocket, I think.

Re: Show HN: Nchan – a pub/sub server as an Nginx module

#40

Post by solyaris »

slact wrote: post #2

This is a huge refactoring of an old project of mine -- the Nginx HTTP Push Module. I'm wondering if anyone here has used it. Most importantly, I want feedback on the documentation. Did I overcomplicate things? Does it need more examples? Does it need more live code? Is it too long? Too short? Etc.

great project! I retwitted: https://twitter.com/solyarisoftware/status/67379850204928000...

Documentation is VERY well done. publisher: Curl examples are perfect. subscribers: maybe some examples in a language ( Ruby? :-) ) could help dummies like me. I'll study and if I can I'll propose you

respect giorgio

Post reply on HN