Live data from Hacker News

How much faster is Redis at storing a blob of JSON compared to Postgres?

peterbe.com

11–20 of 119 posts

Re: How much faster is Redis at storing a blob of JSON compared to Postgres?

#11
Something is very wrong for the median read of what should be a moderate sized blob with an indexed lookup to be 8ms.

The article doesn’t give enough details to really understand what is that is happening here, but I would recommend everyone to take this 14/16x report and not expect that to be the general case.

Re: How much faster is Redis at storing a blob of JSON compared to Postgres?

#12

Why can't Postgres implement a transparent caching layer similar to Redis?

There is a query cache. But I sorta doubt this benchmark took appropriate advantage of it. We don't know enough about how the author is running postgres to be sure. But the reported postgres numbers are pretty slow for machine-local SELECTs. Maybe Django's ORM is getting in the way?

Re: How much faster is Redis at storing a blob of JSON compared to Postgres?

#13
post #10

Its faster, but then, thats the point of it. Its like saying that memcache is faster than postgres. they do different things for different purposes. I still wouldn't trust redis for anything other than ephemeral storage. Most of the use cases where we use redis assume that the data will at some point go away. The places where we use postgres assume that data is permanent. Infact, we use both together, and it works re…

The article talks about _how much_ it is faster which is the interesting question. For a small project you might not want the added complexity of another database and decide to store JSON blobs in PostgreSQL. In this case it's important to understand the performance trade-off you're making.

In Django it's literally a matter of adding 5 lines of code to enable the redis cache. The complexity is in actually setting up the Redis server, but this can also be trivial in many cases.

Re: How much faster is Redis at storing a blob of JSON compared to Postgres?

#15

Its faster, but then, thats the point of it. Its like saying that memcache is faster than postgres. they do different things for different purposes. I still wouldn't trust redis for anything other than ephemeral storage. Most of the use cases where we use redis assume that the data will at some point go away. The places where we use postgres assume that data is permanent. Infact, we use both together, and it works re…

It's faster, but only under the specific design condition of both having to wait for storage to finish and not caring about differences between the presented storage options.

If the design condition allows asynchronous operation and I don't need to wait for background tasks to finish before I reply, it's not faster unless I somehow care more about background CPU time per request than I do about things like storage guarantees.

If the design condition requires durable storage to have happened before I reply, putting it in Redis doesn't even count as getting started.

Re: How much faster is Redis at storing a blob of JSON compared to Postgres?

#17
By default Django closes PostgreSQL connections after handling each web request. I’m not certain but what we’re probably seeing here in latency is mostly the substantial overhead in setting up a PG connection. Redis will still be faster but I’d guess the gap would be significantly tighter with persistent connections enabled.

Re: How much faster is Redis at storing a blob of JSON compared to Postgres?

#18

Its faster, but then, thats the point of it. Its like saying that memcache is faster than postgres. they do different things for different purposes. I still wouldn't trust redis for anything other than ephemeral storage. Most of the use cases where we use redis assume that the data will at some point go away. The places where we use postgres assume that data is permanent. Infact, we use both together, and it works re…

One of the shops I worked in used Redis as the primary datastore. While there was a lot of extra complexity around relational data (especially referential integrity), the snapshotting systems worked very well and we didn't have to worry about data loss. I don't necessarily think this was an ideal setup (I can't tell you how many headaches Postgres would have solved for us), but I think there are plenty of cases where…

> but I think there are plenty of cases where you could use it without fear that your data will evaporate.

What about "redis-cli flushall"? (Don't type this on your production cluster).

Re: How much faster is Redis at storing a blob of JSON compared to Postgres?

#20
post #13
post #10

Earlier quoted context omitted.

The article talks about _how much_ it is faster which is the interesting question. For a small project you might not want the added complexity of another database and decide to store JSON blobs in PostgreSQL. In this case it's important to understand the performance trade-off you're making.

In Django it's literally a matter of adding 5 lines of code to enable the redis cache. The complexity is in actually setting up the Redis server, but this can also be trivial in many cases.

Sorry, no. The complexity is that you are now relying on another server which can break in new ways, requires startup and shutdown and configuration and possibly load-balancing; management of all the above... updates, security reviews, and documentation. Will you need new server types with an emphasis on RAM? How many and when? How are you monitoring it for errors and performance?

That's the difference between thinking about it as a solution to a problem and thinking about it as part of your infrastructure.

Post reply on HN