Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An update on Redis Streams development

antirez.com

Post reply on HN
Print view View original

21–30 of 48 posts

Re: An update on Redis Streams development

#21

Post by jrajav »

antirez wrote: post #13
acidus wrote: post #5

I wonder if this impacts the plan of releasing Disque as a plugin in Redis 4.2. I always thought that Disque could have a great impact in the field of job queues.

Hello, no change... The original plan was: 4.0 (done) -> Streams back ported to 4.0 (Work in progress) -> 4.2 (or 5.0) with Disque + Cluster improvements + Modules improvements, ... It's just a renaming: 4.0 + Streams backported is now called 5.0 What was to be 4.2 is going to be called 6.0 Why I'm choosing to go for integer numbers? Because I believe that things like 4.2 should be for minor improvements, mostly oper…

Hi, since you're here just a quick question not really worth a Github issue - what is the reason for the 1-second resolution in the DELAY parameter in Disque and will that ever get more fine? We currently use Bull as a job queue on Redis and the delay is a key and visible part of our application, so that alone kind of eliminates Disque as a consideration for us.

Re: An update on Redis Streams development

#22

Post by j_s »

antirez wrote: post #15
neovintage wrote: post #12

I believe Redis has a great developer experience. It’s easy to get set up and use. When I see the work on Redis streams, I think its going bring a much better getting started experience for developers that want to start using evented architectures. This might be a turning point where we see more developers and applications utilizing those types of architectures. In the end, this might give Apache Kafka a run for its…

Thanks for your interesting POV on the Redis/Kafka intersection on this. What could be interesting is that while Redis Streams are certainly totally inspired from Kafka streams, it's just a conceptual thing, so the two things can act in very different ways in practice. For instance just imagine that I never touched a Kafka system in my life, never used it, don't even know the API, I only read all the documentation th…

I'm a big fan and I hope you see this comment before the edit window passes (unlikely, unfortunately):

You need to get yourself some whitespace in this reply!

Re: An update on Redis Streams development

#23

Post by antirez »

j_s wrote: post #22
antirez wrote: post #15

Earlier quoted context omitted.

Thanks for your interesting POV on the Redis/Kafka intersection on this. What could be interesting is that while Redis Streams are certainly totally inspired from Kafka streams, it's just a conceptual thing, so the two things can act in very different ways in practice. For instance just imagine that I never touched a Kafka system in my life, never used it, don't even know the API, I only read all the documentation th…

I'm a big fan and I hope you see this comment before the edit window passes (unlikely, unfortunately): You need to get yourself some whitespace in this reply!

Thanks! Done :-)

Re: An update on Redis Streams development

#24

Post by viperscape »

luhn wrote: post #6
netgusto wrote: post #2

Could someone ELI5 what's a redis stream ? I thought that pub/sub mecanisms were some kind of a stream already.

Redis Pubsub is fire-and-forget, so if you aren't listening when a message is fired, you'll never receive it. Redis Streams store messages, so you can connect and read all the messages since you last checked. It's a similar model to Kafka.

is it strictly in-mem? for instance if redis is restarted is there an option for unread streams to be saved with redis persistence

Re: An update on Redis Streams development

#25

Post by antirez »

jrajav wrote: post #21
antirez wrote: post #13

Earlier quoted context omitted.

Hello, no change... The original plan was: 4.0 (done) -> Streams back ported to 4.0 (Work in progress) -> 4.2 (or 5.0) with Disque + Cluster improvements + Modules improvements, ... It's just a renaming: 4.0 + Streams backported is now called 5.0 What was to be 4.2 is going to be called 6.0 Why I'm choosing to go for integer numbers? Because I believe that things like 4.2 should be for minor improvements, mostly oper…

Hi, since you're here just a quick question not really worth a Github issue - what is the reason for the 1-second resolution in the DELAY parameter in Disque and will that ever get more fine? We currently use Bull as a job queue on Redis and the delay is a key and visible part of our application, so that alone kind of eliminates Disque as a consideration for us.

Hello, thanks I'll take this in mind. The resolution could be made to be accepted in milliseconds, and it should be simple to honor it with an error for example of 50 milliseconds or alike, but to get true 1 ms resolution requires non trivial changes to the system that must act like a real time scheduler in some way...

Re: An update on Redis Streams development

#26

Post by j_s »

antirez wrote: post #17
pselbert wrote: post #16

I must admit, for the past couple months I've been digging for status info about Redis Streams. They would fit a use case we have perfectly, but we use cloud providers for Redis so manual compilation with modules isn't possible. Really pleased to see that I'm not the only one digging for info and that work is ongoing.

Thank you, I believe it's my fault that many potential users remained wondering. Sometimes I forget that the world is not inside Twitter... and I should instead blog more and tweet less, both for myself, because writing a blog post gives me much more sense to accomplish something, and for people interested in Redis that will find information more readily, and also searching via Google and so forth.

Why not both?!! (summarizing tweets occasionally)

Perhaps a resurrection of the Redis Watch newsletter is in order! Are there any existing alternatives?

Re: An update on Redis Streams development

#27

Post by mdomans »

Here's a question from an old Redis hater (the note is important, since my question is going to be slightly biased - I disagree with a lot of core decisions behind Redis):

How is this going to be different from Kafka? And I don't mean implementation details, because these are always fun read. Kafka is on the market for ~7 years, during which it has proven to be oh-so-fast and pretty durable.

Oh, and while I'm at it. Here's another problem Redis geniousl added: a GIL. GIL is a great idea, but comes with huge tradeoffs. David Beazley spent years showing how many tricks you can play upon yourself with GIL.

So ... now you have Streams and GIL together. And you already have dicts (you call them hashmaps). I have a feeling you're trying to implement Python. If so, it's done. But come on, 3.6 is cool. And we're kinda solving the GIL problem. With PyPy. Which will blow your mind.

Re: An update on Redis Streams development

#28

Post by antirez »

mdomans wrote: post #27

Here's a question from an old Redis hater (the note is important, since my question is going to be slightly biased - I disagree with a lot of core decisions behind Redis): How is this going to be different from Kafka? And I don't mean implementation details, because these are always fun read. Kafka is on the market for ~7 years, during which it has proven to be oh-so-fast and pretty durable. Oh, and while I'm at it.…

So yes, that's it. The Redis hidden agenda was to compete with Python (to be honest not a big secret... you can see that Redis works internally as an interpreter in a pretty obvious way), and now that you uncovered it, I'm going to say it aloud: we are going to exit in a few months with a new package system we are working at for 5 years at this point, based on the blockchain (proof of installation), which will kill NPM completely, so with a Python killer + an NPM killer we'll see what mind will be blown.

Re: An update on Redis Streams development

#29

Post by kahlonel »

mdomans wrote: post #27

Here's a question from an old Redis hater (the note is important, since my question is going to be slightly biased - I disagree with a lot of core decisions behind Redis): How is this going to be different from Kafka? And I don't mean implementation details, because these are always fun read. Kafka is on the market for ~7 years, during which it has proven to be oh-so-fast and pretty durable. Oh, and while I'm at it.…

How can you compare a programming language and a memory store? How are they even remotely comparable? Redis needs a syntax to exchange data with clients in a human-readable way.

Honest question: Why do you hate Redis?

Re: An update on Redis Streams development

#30

Post by Redsquare »

antirez wrote: post #28
mdomans wrote: post #27

Here's a question from an old Redis hater (the note is important, since my question is going to be slightly biased - I disagree with a lot of core decisions behind Redis): How is this going to be different from Kafka? And I don't mean implementation details, because these are always fun read. Kafka is on the market for ~7 years, during which it has proven to be oh-so-fast and pretty durable. Oh, and while I'm at it.…

So yes, that's it. The Redis hidden agenda was to compete with Python (to be honest not a big secret... you can see that Redis works internally as an interpreter in a pretty obvious way), and now that you uncovered it, I'm going to say it aloud: we are going to exit in a few months with a new package system we are working at for 5 years at this point, based on the blockchain (proof of installation), which will kill N…

great response
Post reply on HN