Live data from Hacker News

How much faster is Redis at storing a blob of JSON compared to Postgres?

peterbe.com

21–30 of 119 posts

Re: How much faster is Redis at storing a blob of JSON compared to Postgres?

#21
No index, not the same size of datastore, queries in a loop the same entry, no cache config on the db side, only one hydrates an ORM... Don't get confused, it's not because data comes from a database that it comes from the disk when you query it.

I'm pretty sure we can go to a x3 max for the db.

Re: How much faster is Redis at storing a blob of JSON compared to Postgres?

#23

Something is very wrong for the median read of what should be a moderate sized blob with an indexed lookup to be 8ms. The article doesn’t give enough details to really understand what is that is happening here, but I would recommend everyone to take this 14/16x report and not expect that to be the general case.

Looking up a row of JSONB data by a primary key in a 3GB PostgreSQL table on my laptop takes 0.4-0.5ms, since it easily fits in memory and Postgres caches the data.

Re: How much faster is Redis at storing a blob of JSON compared to Postgres?

#24

Its faster, but then, thats the point of it. Its like saying that memcache is faster than postgres. they do different things for different purposes. I still wouldn't trust redis for anything other than ephemeral storage. Most of the use cases where we use redis assume that the data will at some point go away. The places where we use postgres assume that data is permanent. Infact, we use both together, and it works re…

Couchbase really is the best of both worlds, in one product. Store in memory first (memcached) and disk later, on 0 or more cluster nodes. From the calling API, you can choose whether to block or not for the disk commit.

Edit: typos

Re: How much faster is Redis at storing a blob of JSON compared to Postgres?

#25

Alas, there's not enough context or code here to even really know what's going on (or maybe I just don't know Django well enough to guess at what isn't said). It sounds like the author is comparing storing and retrieving a record with a JSON field in a PostgreSQL table versus storing and retrieving a raw string (the raw JSON) in Redis? So it's no surprise that the Redis solution is faster. Besides parsing and indexin…

I guess the answer to does apple smell more apple than orange is yes

Re: How much faster is Redis at storing a blob of JSON compared to Postgres?

#26

Something is very wrong for the median read of what should be a moderate sized blob with an indexed lookup to be 8ms. The article doesn’t give enough details to really understand what is that is happening here, but I would recommend everyone to take this 14/16x report and not expect that to be the general case.

I think the author suggested that the cause of some of the latency might be the processing in Python as data goes through the ORM. I like the Django ORM but it does not prioritize speed. It's possible that Python time plus the ORM doing something stupid (perhaps multiple round trips to the DB, reconnects, or weird instrumentation in the DB router or manager) could account for those extra milliseconds.

Re: How much faster is Redis at storing a blob of JSON compared to Postgres?

#28
Is this really that surprising?

Postgres is a RDMS, with transactional support and everything. While Redis is powerful, it is much simpler and focused, not nearly as comprehensive as Postgres.

TL;DR Postgres is doing a lot more heavy-lifting than Redis, it feels slow because that much of work is not necessary for simple kv lookup

Re: How much faster is Redis at storing a blob of JSON compared to Postgres?

#30

Alas, there's not enough context or code here to even really know what's going on (or maybe I just don't know Django well enough to guess at what isn't said). It sounds like the author is comparing storing and retrieving a record with a JSON field in a PostgreSQL table versus storing and retrieving a raw string (the raw JSON) in Redis? So it's no surprise that the Redis solution is faster. Besides parsing and indexin…

A reasonably configured Postgres is easily sub-millisecond for reads and writes on a small dataset like this. I suspect that the author is doing something quite wrong.

People generally reach for Redis too quickly, IMO. Postgres is about as fast, has better tooling, and will slow down gracefully as the dataset increases, rather than evicting records as the limit is hit.

That said, I guess if you're relatively junior, it's hard to grossly misconfigure Redis, so that's a point in its favor. Somewhat like Java vs Node.js--you can write higher performance network services in Java, but it's a lot easier to make mistakes.

Post reply on HN