Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis 8.8: New array data structure, rate limiter, performance improvements

redis.io

Post reply on HN
Print view View original

51–60 of 118 posts

Re: Redis 8.8: New array data structure, rate limiter, performance improvements

#51

Post by echelon »

9dev wrote: post #3

While I love Redis as a versatile tool for external data structures, it's still lacking in two areas IMHO: One, it would be cool to be able to embed it, similar to sqlite, directly into applications. Two, the HA story is so much more complicated than it should be. I totally acknowledge that concurrency and distributed computing is hard, but it should not require reading heaps of documentation and understanding two en…

> it's still lacking in two areas

This is entirely different than what Redis is and tries to solve.

Sqlite is embedded. It's not a distributed SQL. Redis is a distributed data structure store and concurrency primitive. These are worlds apart.

> HA story is so much more complicated than it should be

It is precisely as complicated as it needs to be. You don't want data loss.

If you're in the business of high available fault tolerance, you read the manual and learn how to Redis.

Re: Redis 8.8: New array data structure, rate limiter, performance improvements

#53

Post by zozbot234 »

focusgroup0 wrote: post #18

given his ds4 project, likely collaborated with DeepSeek for this release: https://github.com/antirez/ds4

The experimental SSD streaming feature (author's demo @ https://x.com/antirez/status/2062536214675067322 - recently merged into the main branch) is great news for that project, allowing for SOTA inference (DeepSeek V4 Flash and Pro!) on RAM-limited machines. Now we need work on large-ish scale batching in order to recover tok/s under the SSD streaming scenario. It's not helpful when running normally (at least not on Apple Silicon) since thermal/power throttling is the constraint in that case, but SSD streaming is a whole other consideration.

Re: Redis 8.8: New array data structure, rate limiter, performance improvements

#56

Post by trumpdong »

9dev wrote: post #42
olavgg wrote: post #40

Earlier quoted context omitted.

But that is his point. If you cannot find the session id in redis, you login again. If your Redis server crash, you start a new one and everyone just login again. No data is lost.

Sure the data is lost. A session commonly holds arbitrary state, and even if it’s just the login information. This is ridiculous.

If you consider it important, you have to store it in a real database. No buts. If you don't consider it important, sharded redis works fine.

Re: Redis 8.8: New array data structure, rate limiter, performance improvements

#57

Post by 9dev »

echelon wrote: post #51
9dev wrote: post #3

While I love Redis as a versatile tool for external data structures, it's still lacking in two areas IMHO: One, it would be cool to be able to embed it, similar to sqlite, directly into applications. Two, the HA story is so much more complicated than it should be. I totally acknowledge that concurrency and distributed computing is hard, but it should not require reading heaps of documentation and understanding two en…

> it's still lacking in two areas This is entirely different than what Redis is and tries to solve. Sqlite is embedded. It's not a distributed SQL. Redis is a distributed data structure store and concurrency primitive. These are worlds apart. > HA story is so much more complicated than it should be It is precisely as complicated as it needs to be. You don't want data loss. If you're in the business of high available…

What kind of an answer is that? This software is perfect the way it is, you’re just to inept to hold it right?

A high availability protocol should not leak into the client. It should be able to discover other nodes. It should not land in broken states so easily. It should not limit the number of writers. It should not error during failover.

Are these hard problems? Yes. Should we just accept that things are hard because that’s how the gods have given them to us? No.

Re: Redis 8.8: New array data structure, rate limiter, performance improvements

#58

Post by willempienaar »

simonw wrote: post #50

> Rate limiting is one of the most common Redis use cases. Traditionally, users implemented rate limiters using server-side Lua scripts combined with client logic. In Redis 8.8, we introduce a window counter rate limiter (by @raffertyyu, together with the Redis team). I had a look for this and it turns out it's slightly mis-described there - it's not a window counter, it's a "GCRA (Generic Cell Rate Algorithm)" - a l…

Also, the “cell” in Generic Cell Rate Algorithm is an ATM cell. GCRA is 1990s telecom, the scheduling algorithm ATM switches used to check that 53-byte cells were arriving on the wire at the agreed rate.

Re: Redis 8.8: New array data structure, rate limiter, performance improvements

#59

Post by 9dev »

trumpdong wrote: post #56
9dev wrote: post #42

Earlier quoted context omitted.

Sure the data is lost. A session commonly holds arbitrary state, and even if it’s just the login information. This is ridiculous.

If you consider it important, you have to store it in a real database. No buts. If you don't consider it important, sharded redis works fine.

Redis is a real database. If I wasn’t convinced it could retain data I hand it, I wouldn’t use it in the first place.

Just because it works for your use case right now doesn’t mean there isn’t room for improvements to support others too.

Re: Redis 8.8: New array data structure, rate limiter, performance improvements

#60

Post by marklubi »

amtamt wrote: post #4
9dev wrote: post #3

While I love Redis as a versatile tool for external data structures, it's still lacking in two areas IMHO: One, it would be cool to be able to embed it, similar to sqlite, directly into applications. Two, the HA story is so much more complicated than it should be. I totally acknowledge that concurrency and distributed computing is hard, but it should not require reading heaps of documentation and understanding two en…

Genuinely interested why we need HA in redis, just not read round robin from multiple non-HA instances? Redis (and memcache) are memory caches and should be treated like that, not like highly consistent distributed session store.

For the project I've been working on for more than 15 years, we make extensive use of the pub/sub functionality for distributing live data. Pub/sub scales well across the cluster. Publish to one, and it goes out to subscribers on any of the nodes that they've connected to.

Will millions of users, high availability is critical for this functionality.

Post reply on HN