61–70 of 118 posts
While I love Redis as a versatile tool for external data structures, it's still lacking in two areas IMHO: One, it would be cool to be able to embed it, similar to sqlite, directly into applications. Two, the HA story is so much more complicated than it should be. I totally acknowledge that concurrency and distributed computing is hard, but it should not require reading heaps of documentation and understanding two en…
What would be the point of embedding Redis into an application? What's the advantage of using Redis over using the builtin (or third party) data structures of the language the application is developed in? I'm asking as a non-webdev who never quite got what Redis actually does, but would love to learn.
Embedding would make local dev/CI integration testing convenient.
Embedding replicated Redis with each application instance would give you HA benefits while infra-management complexity.
Embedded redis (even via local RPC) is still going to be faster than a lot of languages or frameworks’ built-in data structures. Large array operations in, say, Python are gonna slower than RPCing to Redis (assuming that the data structures are built gradually and not built all at once); to beat Redis you’d have to use numpy or something—-which is definitely preferable, but is extra work if your app already uses Redis for other things.
Just like choosing SQLite over e.g. LMDB or RocksDB, embedded Redis would be a nice future proofing option for small apps during the prototype phase; less would have to be changed to move Redis out of the app than if a different cache or persistence service were chosen.
Earlier quoted context omitted.
If you consider it important, you have to store it in a real database. No buts. If you don't consider it important, sharded redis works fine.
Redis is a real database. If I wasn’t convinced it could retain data I hand it, I wouldn’t use it in the first place. Just because it works for your use case right now doesn’t mean there isn’t room for improvements to support others too.
Oh good, then you don't need to do any of the stuff that you suggested to do
Earlier quoted context omitted.
This case is exactly what he talks about. To get HA just setup more than one redis cache - or rebuild the session if it was lost in the redis cache.
It’s not. Imagine a web app that stores your user information in a session store, mapped by your cookie-provided session ID. Your web app searches redis 1 for the session id, but since that key is on redis 2, the lookup fails and the application thinks there is no such session, and rejects the request. Now you could solve this specific case by sharding by prefix, or by querying all instances, but then you still do no…
The app would look up in both databases. If it exists in any, there would be a session.
Thisnis strictly different from partitioning which I think you are mixing it up with.
Paritioning is for performance not HA
Earlier quoted context omitted.
But that is his point. If you cannot find the session id in redis, you login again. If your Redis server crash, you start a new one and everyone just login again. No data is lost.
Sure the data is lost. A session commonly holds arbitrary state, and even if it’s just the login information. This is ridiculous.
You, obviously, don't commit important data only to a session that you can loose, if the application does not allow it.
We use redis as infrastructure. To route events and as a cache.
For us redis could go down and we would merely see a degradation of our service with no data loss.
I recommend using redis like that. And then use a database that supports transactions for real data problems.
But we are different. And that's OK.
Earlier quoted context omitted.
It’s not. Imagine a web app that stores your user information in a session store, mapped by your cookie-provided session ID. Your web app searches redis 1 for the session id, but since that key is on redis 2, the lookup fails and the application thinks there is no such session, and rejects the request. Now you could solve this specific case by sharding by prefix, or by querying all instances, but then you still do no…
I don't think you understand what HA means. The app would look up in both databases. If it exists in any, there would be a session. Thisnis strictly different from partitioning which I think you are mixing it up with. Paritioning is for performance not HA
Earlier quoted context omitted.
What would be the point of embedding Redis into an application? What's the advantage of using Redis over using the builtin (or third party) data structures of the language the application is developed in? I'm asking as a non-webdev who never quite got what Redis actually does, but would love to learn.
To me the thing I like about Redis is that it gives you a storage engine very suitable for caches; it handles TTLs and memory pressure, as well as built-in serialization with the ability to get better performance by allowing for some data loss. At the same time, many users will be deploying small programs to individual machines. If you could just have Redis be embedded this would make it very operationally simple: no…
If you use multiple nodes, then you probably want your redis lifecycle not be tied to application lifecycle.
Earlier quoted context omitted.
To me the thing I like about Redis is that it gives you a storage engine very suitable for caches; it handles TTLs and memory pressure, as well as built-in serialization with the ability to get better performance by allowing for some data loss. At the same time, many users will be deploying small programs to individual machines. If you could just have Redis be embedded this would make it very operationally simple: no…
Well, if you have a single instance than using language libraries and structures will be better in most cases. If you use multiple nodes, then you probably want your redis lifecycle not be tied to application lifecycle.
Earlier quoted context omitted.
It doesn’t seem like the right tool for the job, though. Aren’t your own programming language’s constructs much more well-defined / understood ?
Redis has some pretty useful primitive that many languages don't: - HyperLogLog, bloom filter, other probabilistic data structures - Geospatial operations on stored points and polygons - Expiring keys, for creating caches These aren't in most standard libraries, and the Redis implementations tend to be fast, robust and well understood.
Every language you can talk to redis most likely has a library to do that, and it probably works much better with the rest of application than "embedded redis". If it doesn't, it probably has C-FFI and there is "fast, robust and well understood" implementations in C.
Earlier quoted context omitted.
For simple cases, it is probably a total overkill to even consider it, but for something heavier, embedding the database gives you a chance to trivially migrate later to a separate database server.
Redis is not a database. It’s a key / value store.