RCP 11 – Stream data type proposal for Redis
1–10 of 18 posts
Re: RCP 11 – Stream data type proposal for Redis
#2Salvatore is a machine. Since 2012 Redis has been a main tool in my preverbal belt and it has always seemed to have forward momentum.
Re: RCP 11 – Stream data type proposal for Redis
#3Yes, please. I've never understood why someone wants to do pub/sub and only base the reliability on TCP/WebSockets. The concept of "fire and hope everyone who wants the message still has a connection open" always seemed fragile to me.
This has been the reason I've recommended implementing long-polling instead of WebSockets for real-time applications. And every time I see a real-time solution which only uses WebSockets I try to steer away from it. Once you have a reliable data model (which includes log position, retrieving old messages etc.) it's just as simple to implement long-polling as WebSockets. With WebSockets-only solution I can't help but think they base all the message delivery reliability on TCP.
This proposal looks like a clean (very Redis-like) solution, and I immediately see use cases for it.
Re: RCP 11 – Stream data type proposal for Redis
#4Yes, please. I've never understood why someone wants to do pub/sub and only base the reliability on TCP/WebSockets. The concept of "fire and hope everyone who wants the message still has a connection open" always seemed fragile to me. This has been the reason I've recommended implementing long-polling instead of WebSockets for real-time applications. And every time I see a real-time solution which only uses WebSocket…
Did you look into the EventSource API? It's basically automatic long-polling with a simple file format built into the browser. It supports pretty much every "real-time" pattern: regular polling, long polling, and streaming.
The browser handles the log position for you (via the id field and the Last-Event-ID header) and automatically reconnects when the server closes the stream or the connection is lost.
The proposed Redis API seems to fit extremely well to this model. My previous usage of EventSource with Redis worked by sending the entire state whenever someone (re-)connects and using PubSub afterwards. This works well for me, but likely doesn't scale very well.
Re: RCP 11 – Stream data type proposal for Redis
#5Salvatore is a machine. Since 2012 Redis has been a main tool in my preverbal belt and it has always seemed to have forward momentum.
Have you paid for Redis? If so, was it direct donation, Redis Labs services, or committing code?
Edit: Not asking to be a jerk or insist that s/he donate. I use lots of things heavily that I've never contributed to. I'm just curious how the Redis community works.
Re: RCP 11 – Stream data type proposal for Redis
#6That's cool. In case you haven't seen this blog post [1], it provides an extended discussion of the uses of this abstraction.
One comment about the groups API is that while it is very convenient, it seems a bit fragile - if a consumer is nuked immediately after it gets a fresh batch from redis then this batch is lost forever.
[1] https://engineering.linkedin.com/distributed-systems/log-wha....
Re: RCP 11 – Stream data type proposal for Redis
#7[deleted]
Re: RCP 11 – Stream data type proposal for Redis
#8Yes, please. I've never understood why someone wants to do pub/sub and only base the reliability on TCP/WebSockets. The concept of "fire and hope everyone who wants the message still has a connection open" always seemed fragile to me. This has been the reason I've recommended implementing long-polling instead of WebSockets for real-time applications. And every time I see a real-time solution which only uses WebSocket…
Did you look into the EventSource API? It's basically automatic long-polling with a simple file format built into the browser. It supports pretty much every "real-time" pattern: regular polling, long polling, and streaming. The browser handles the log position for you (via the id field and the Last-Event-ID header) and automatically reconnects when the server closes the stream or the connection is lost. The proposed…
Server-Sent Event (SSE) is very underrated HTML5 technology.
If you dont need bi-directional realtime messaging, use SSE.
Re: RCP 11 – Stream data type proposal for Redis
#9Yes, please. I've never understood why someone wants to do pub/sub and only base the reliability on TCP/WebSockets. The concept of "fire and hope everyone who wants the message still has a connection open" always seemed fragile to me. This has been the reason I've recommended implementing long-polling instead of WebSockets for real-time applications. And every time I see a real-time solution which only uses WebSocket…
Did you look into the EventSource API? It's basically automatic long-polling with a simple file format built into the browser. It supports pretty much every "real-time" pattern: regular polling, long polling, and streaming. The browser handles the log position for you (via the id field and the Last-Event-ID header) and automatically reconnects when the server closes the stream or the connection is lost. The proposed…
The only big barrier to adopting the EventSource API is the complete lack of support from Microsoft's browsers: http://caniuse.com/#feat=eventsource
Re: RCP 11 – Stream data type proposal for Redis
#10Yes, please. I've never understood why someone wants to do pub/sub and only base the reliability on TCP/WebSockets. The concept of "fire and hope everyone who wants the message still has a connection open" always seemed fragile to me. This has been the reason I've recommended implementing long-polling instead of WebSockets for real-time applications. And every time I see a real-time solution which only uses WebSocket…
This is exactly what we built for Poll Everywhere with http://firehose.io/ to push updates in realtime reliably over crappy wifi connections. Since we're used by presenters at huge conferences with dicey networks we had to spend a lot of time thinking through all sorts of low quality connection scenarios.
It's exciting to see a proposal for an all-Redis implementation.