“Memory is faster than disk, more at 11”
Also, treating json like a string is faster than treating it like an object. I'd be curious to see the Postgres numbers with the json in a varchar or similar.
How much faster is Redis at storing a blob of JSON compared to Postgres?
31–40 of 119 posts
Re: How much faster is Redis at storing a blob of JSON compared to Postgres?
#32Earlier quoted context omitted.
In Django it's literally a matter of adding 5 lines of code to enable the redis cache. The complexity is in actually setting up the Redis server, but this can also be trivial in many cases.
Sorry, no. The complexity is that you are now relying on another server which can break in new ways, requires startup and shutdown and configuration and possibly load-balancing; management of all the above... updates, security reviews, and documentation. Will you need new server types with an emphasis on RAM? How many and when? How are you monitoring it for errors and performance? That's the difference between thinki…
True, but a lot of that can be solved with PaaS/DBaaS products like compose or something
Re: How much faster is Redis at storing a blob of JSON compared to Postgres?
#33Re: How much faster is Redis at storing a blob of JSON compared to Postgres?
#34Alas, there's not enough context or code here to even really know what's going on (or maybe I just don't know Django well enough to guess at what isn't said). It sounds like the author is comparing storing and retrieving a record with a JSON field in a PostgreSQL table versus storing and retrieving a raw string (the raw JSON) in Redis? So it's no surprise that the Redis solution is faster. Besides parsing and indexin…
As he chose to show us Django ORM code, instead of the postgres table definition, I'm guessing he's not exactly a postgres wizard. It reads to me like to me like he's writing to a normalized table (complete with foreign key constraints and likely indexes) and comparing against writing a single serialized blob to to a Redis key.
I too would expect the performance improvement to be significantly faster. This guy probably doesn't know how to benchmark his code, either.
Re: How much faster is Redis at storing a blob of JSON compared to Postgres?
#35Earlier quoted context omitted.
One of the shops I worked in used Redis as the primary datastore. While there was a lot of extra complexity around relational data (especially referential integrity), the snapshotting systems worked very well and we didn't have to worry about data loss. I don't necessarily think this was an ideal setup (I can't tell you how many headaches Postgres would have solved for us), but I think there are plenty of cases where…
> but I think there are plenty of cases where you could use it without fear that your data will evaporate. What about "redis-cli flushall"? (Don't type this on your production cluster).
What about 'drop database' for Postgres?
Re: How much faster is Redis at storing a blob of JSON compared to Postgres?
#36By default Django closes PostgreSQL connections after handling each web request. I’m not certain but what we’re probably seeing here in latency is mostly the substantial overhead in setting up a PG connection. Redis will still be faster but I’d guess the gap would be significantly tighter with persistent connections enabled.
Really? It doesn't use a connection pool?
Re: How much faster is Redis at storing a blob of JSON compared to Postgres?
#37Something is very wrong for the median read of what should be a moderate sized blob with an indexed lookup to be 8ms. The article doesn’t give enough details to really understand what is that is happening here, but I would recommend everyone to take this 14/16x report and not expect that to be the general case.
I think the author suggested that the cause of some of the latency might be the processing in Python as data goes through the ORM. I like the Django ORM but it does not prioritize speed. It's possible that Python time plus the ORM doing something stupid (perhaps multiple round trips to the DB, reconnects, or weird instrumentation in the DB router or manager) could account for those extra milliseconds.
It's also in a json type in Postgres rather than a blob or varchar, while it's just raw data in redis.
Re: How much faster is Redis at storing a blob of JSON compared to Postgres?
#38Alas, there's not enough context or code here to even really know what's going on (or maybe I just don't know Django well enough to guess at what isn't said). It sounds like the author is comparing storing and retrieving a record with a JSON field in a PostgreSQL table versus storing and retrieving a raw string (the raw JSON) in Redis? So it's no surprise that the Redis solution is faster. Besides parsing and indexin…
A reasonably configured Postgres is easily sub-millisecond for reads and writes on a small dataset like this. I suspect that the author is doing something quite wrong. People generally reach for Redis too quickly, IMO. Postgres is about as fast, has better tooling, and will slow down gracefully as the dataset increases, rather than evicting records as the limit is hit. That said, I guess if you're relatively junior,…
> That said, I guess if you're relatively junior, it's hard to grossly misconfigure Redis, so that's a point in its favor.
Redis persistence (or possibly lack-thereof) is probably quite complicated for a junior developer to understand. I'd actually expect a better outcome with PSQL.
Re: How much faster is Redis at storing a blob of JSON compared to Postgres?
#39FWIW there is an update to this article at - https://www.peterbe.com/plog/update-to-speed-comparison-for-... TLDR; this update is not about redis vs postgresql but about caching!
If it were about caching, he wouldn't have used an ORM and a json Postgres type. And there would have been more detail about the query cache, backing store, and wal+fsync settings for Postgres.
Re: How much faster is Redis at storing a blob of JSON compared to Postgres?
#40“Memory is faster than disk, more at 11”
yeah and in normal operation the OS/PG will have all your data in memory (or all your indexes if very, very much data or under spec'd db server).
Disk is for writes.