Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RCP 11 – Stream data type proposal for Redis

github.com

Post reply on HN
Print view View original

11–18 of 18 posts

Re: RCP 11 – Stream data type proposal for Redis

#11

Post by lobster_johnson »

judofyr wrote: post #3

Yes, please. I've never understood why someone wants to do pub/sub and only base the reliability on TCP/WebSockets. The concept of "fire and hope everyone who wants the message still has a connection open" always seemed fragile to me. This has been the reason I've recommended implementing long-polling instead of WebSockets for real-time applications. And every time I see a real-time solution which only uses WebSocket…

Take a look at SSE (Server-Sent Events, aka EventSource). Unlike WebSockets, the it has the concept of an "event ID", which allows the protocol to automatically continue from the last event, no extra roundtrip needed to send a setup message to the server. Implemented by every major browser (except, of course, IE, where you can polyfill it pretty easily).

Re: RCP 11 – Stream data type proposal for Redis

#12

Post by lobster_johnson »

bradgessler wrote: post #9
luchs wrote: post #4

Earlier quoted context omitted.

Did you look into the EventSource API? It's basically automatic long-polling with a simple file format built into the browser. It supports pretty much every "real-time" pattern: regular polling, long polling, and streaming. The browser handles the log position for you (via the id field and the Last-Event-ID header) and automatically reconnects when the server closes the stream or the connection is lost. The proposed…

The only big barrier to adopting the EventSource API is the complete lack of support from Microsoft's browsers: http://caniuse.com/#feat=eventsource

Fortunately, polyfilling is fairly easy, e.g. https://github.com/Yaffle/EventSource.

Re: RCP 11 – Stream data type proposal for Redis

#14

Post by louthy »

This looks very interesting. I use Redis as a backend for an actor-system [1] as part of a larger functional framework for C# [2]. I use RPUSH to add messages to an actor's queue along with PUBLISH for saying 'new message' to the actor that should check its inbox, this is then followed by LINDEX 0 to peek at the item at the head of the queue, before calling LPOP when the message has been successfully processed. If I'm understanding this correctly, that process could be wrapped up in a single stream?

[1] https://github.com/louthy/language-ext/tree/master/LanguageE...

[2] https://github.com/louthy/language-ext

Re: RCP 11 – Stream data type proposal for Redis

#15

Post by sintaxi »

smt88 wrote: post #5
sintaxi wrote: post #2

Salvatore is a machine. Since 2012 Redis has been a main tool in my preverbal belt and it has always seemed to have forward momentum.

Have you paid for Redis? If so, was it direct donation, Redis Labs services, or committing code? Edit: Not asking to be a jerk or insist that s/he donate. I use lots of things heavily that I've never contributed to. I'm just curious how the Redis community works.

No, only used the open source library on my own.

Re: RCP 11 – Stream data type proposal for Redis

#16

Post by lobster_johnson »

louthy wrote: post #14

This looks very interesting. I use Redis as a backend for an actor-system [1] as part of a larger functional framework for C# [2]. I use RPUSH to add messages to an actor's queue along with PUBLISH for saying 'new message' to the actor that should check its inbox, this is then followed by LINDEX 0 to peek at the item at the head of the queue, before calling LPOP when the message has been successfully processed. If I'…

Yes. Instead of multiple operations, you would get this from a single operation. But instead of modifying the log on every "pop", the operation would merely update a "read position" on behalf of the client. The position semantics would allow you to jump backwards in the log if needed.

Re: RCP 11 – Stream data type proposal for Redis

#17

Post by jon-v »

thesorrow wrote: post #13

Apache Kafka is a beast. Would be awesome to have a light alternative !

Agreed. I'm currently considering Kafka for a project but am hesitant because ideally we need a zero maintenance solution, and Kafka seems to always end up requiring some handholding.

I'm not familiar with the Redis dev process at all. How long might one expect it to take for a feature of this complexity to make it from RCP to production-ready?

Re: RCP 11 – Stream data type proposal for Redis

#18

Post by louthy »

lobster_johnson wrote: post #16
louthy wrote: post #14

This looks very interesting. I use Redis as a backend for an actor-system [1] as part of a larger functional framework for C# [2]. I use RPUSH to add messages to an actor's queue along with PUBLISH for saying 'new message' to the actor that should check its inbox, this is then followed by LINDEX 0 to peek at the item at the head of the queue, before calling LPOP when the message has been successfully processed. If I'…

Yes. Instead of multiple operations, you would get this from a single operation. But instead of modifying the log on every "pop", the operation would merely update a "read position" on behalf of the client. The position semantics would allow you to jump backwards in the log if needed.

Excellent. This is very exciting! (well, for me at least)
Post reply on HN