Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis is fast – I'll cache in Postgres

dizzy.zone

Post reply on HN
Print view View original

151–160 of 311 posts

Re: Redis is fast – I'll cache in Postgres

#151

Post by huflungdung »

Timshel wrote: post #132
ezekiel68 wrote: post #51

> Both postgres and redis are used with the out of the box settings Ugh. I know this gives the illusion of fairness, but it's not how any self-respecting software engineer should approach benchmarks. You have hardware. Perhaps you have virtualized hardware. You tune to the hardware. There simply isn't another way, if you want to be taken seriously. Some will say that in a container-orchestrated environment, tuning go…

> for only about five minutes of LLM query or web search I think I have more trust in the PG defaults that in the output of a LLM or copy pasting some configuration I might not really understand ...

[dead]

Re: Redis is fast – I'll cache in Postgres

#152

Post by motorest »

smacker wrote: post #141

I like using postgres for everything, it lets me simplify infrastructure. But using it as a cache is a bit concerning in terms of reliability, in my opinion. I have witnessed many incidents when DB was considerably degrading. However, thanks to the cache in redis/memcache, a large part of the requests could still be processed with minimal increase in latency. If I were serving cache from the same DB instance, I guess…

> But using it as a cache is a bit concerning in terms of reliability, in my opinion.

This was the very first time I heard anyone even suggest that storing data in Postgres was a concern in terms of reliability, and I doubt you are the only person in the whole world who has access to critical insight onto the matter.

Is it possible that your prior beliefs are unsound and unsubstantiated?

> I have witnessed many incidents when DB was considerably degrading.

This vague anecdote is meaningless. Do you actually have any concrete scenario in mind? Because anyone can make any system "considerably degrading", even Redis, if they make enough mistakes.

Re: Redis is fast – I'll cache in Postgres

#153

Post by kiney »

Neikius wrote: post #111

I skimmed the article, but why is everyone always going for distributed cache? What is wrong with in-memory cache? Lowest latency, fast, easy to implement. Yeah ok, you have 30 million entries? Sure. You need to sync something over multiple nodes? Not sure I would call that a cache.

because PHP scripts started fresh for each request...

Re: Redis is fast – I'll cache in Postgres

#154

Post by motorest »

smacker wrote: post #147
aiisthefiture wrote: post #144

Earlier quoted context omitted.

Select by id is fast. If you’re using it as a cache and not doing select by id then it’s not a cache.

absolutely. But when PG is running out of open connections or has already consumed all available CPU even the simplest query will struggle.

> But when PG is running out of open connections or has already consumed all available CPU even the simplest query will struggle.

I don't think it is reasonable to assume or even believe that connection exhaustion is an issue specific to Postgres. If you take the time to learn about the topic, you won't need to spend too much time before stumbling upon Redis and connection pool exhaustion issues.

Re: Redis is fast – I'll cache in Postgres

#155

Post by baobun »

motorest wrote: post #152
smacker wrote: post #141

I like using postgres for everything, it lets me simplify infrastructure. But using it as a cache is a bit concerning in terms of reliability, in my opinion. I have witnessed many incidents when DB was considerably degrading. However, thanks to the cache in redis/memcache, a large part of the requests could still be processed with minimal increase in latency. If I were serving cache from the same DB instance, I guess…

> But using it as a cache is a bit concerning in terms of reliability, in my opinion. This was the very first time I heard anyone even suggest that storing data in Postgres was a concern in terms of reliability, and I doubt you are the only person in the whole world who has access to critical insight onto the matter. Is it possible that your prior beliefs are unsound and unsubstantiated? > I have witnessed many incid…

No need to be so combative. Take a chill pill, zoom out and look at the reliability of the entire system and its services rather than the db in isolation. If postgres has issues, it can affect the reliability of the service further if it's also running the cache.

Besides, having the cache on separate hardware can reduce the impact on the db on spikes, which can also factor into reliability.

Having more headroom for memory and CPU can mean that you never reach the load where ot turns to service degradation on the same hw.

Obviously a purpose-built tool can perform better for a specific use-case than the swiss army knife. Which is not to diss on the latter.

Re: Redis is fast – I'll cache in Postgres

#156

Post by motorest »

lemagedurage wrote: post #145
dizzyVik wrote: post #47

Earlier quoted context omitted.

There's a reason this is on my blog and not a paper in a journal. This isn't supposed to show the absolute speed of either tool, the benchmark is not set up for that. I do state that redis has more performance on the table in the blog post.

The main issue is that a reader might mistake Redis as a 2X faster postgres. Memory is 1000X faster than disk (SSD) and with network overhead Redis can still be 100X as fast as postgres for caching workloads. Otherwise, the article does well to show that we can get a lot of baseline performance either way. Sometimes a cache is premature optimisation.

> The main issue is that a reader might mistake Redis as a 2X faster postgres. Memory is 1000X faster than disk (SSD) and with network overhead Redis can still be 100X as fast as postgres for caching workloads.

Your comments suggest that you are definitely missing some key insights onto the topic.

If you, like the whole world, consume Redis through a network connection, it should be obvious to you that network is in fact the bottleneck.

Furthermore, using a RDBMS like Postgres may indeed imply storing data in a slower memory. However, you are ignoring the obvious fact that a service such as Postgres also has its own memory cache, and some query results can and are indeed fetched from RAM. Thus it's not like each and every single query forces a disk read.

And at the end of the day, what exactly is the performance tradeoff? And does it pay off to spend more on an in-memory cache like Redis to buy you the performance Delta?

That's why real world benchmarks like this one are important. They help people think through the problem and reassess their irrational beliefs. You may nitpick about setup and configuration and test patterns and choice of libraries. What you cannot refute are the real world numbers. You may argue they could be better if this and that, but the real world numbers are still there.

Re: Redis is fast – I'll cache in Postgres

#157

Post by odie5533 »

motorest wrote: post #143
odie5533 wrote: post #131

Earlier quoted context omitted.

High read/write and low-ish size. Also it's faster.

> High read/write and low-ish size. Also it's faster You posted a vague and meaningless assertion. If you do not have latency numbers and cost differences, you have absolutely nothing to show for, and you failed to provide any rationale that justified even whether any cache is required at all.

At 10k RPS you'll see a significant cost savings with Redis over DynamoDB.

ElastiCache Serverless (Redis/Memcached): Typical latency is 300–500 microseconds (sub-millisecond response)

DynamoDB On-Demand: Typical latency is single-digit milliseconds (usually between 1–10 milliseconds for standard requests)

Re: Redis is fast – I'll cache in Postgres

#158

Post by motorest »

pcthrowaway wrote: post #99
dizzyVik wrote: post #96

Earlier quoted context omitted.

I still end up recommending using postgres though, don't I?

"I'll use postgres" was going to be your conclusion no matter what I guess? I mean what if an actual benchmark showed Redis is 100X as fast as postgres for a certain use case? What are the constraints you might be operating with? What are the characteristics of your workload? What are your budgetary constraints? Why not just write a blog post saying "Unoptimized postgres vs redis for the lazy, running virtualized wit…

> "I'll use postgres" was going to be your conclusion no matter what I guess?

Would it bother you as well if the conclusion was rephrased as "based on my observations, I see no point in rearchitecting the system to improve the performance by this much"?

I think you are too tied to a template solution that not only you don't stop to think why you're using it or even if it is justified at all. Then, when you are faced with observations that challenge your unfounded beliefs, you somehow opt to get defensive? That's not right.

Re: Redis is fast – I'll cache in Postgres

#160

Post by abtinf »

motorest wrote: post #152
smacker wrote: post #141

I like using postgres for everything, it lets me simplify infrastructure. But using it as a cache is a bit concerning in terms of reliability, in my opinion. I have witnessed many incidents when DB was considerably degrading. However, thanks to the cache in redis/memcache, a large part of the requests could still be processed with minimal increase in latency. If I were serving cache from the same DB instance, I guess…

> But using it as a cache is a bit concerning in terms of reliability, in my opinion. This was the very first time I heard anyone even suggest that storing data in Postgres was a concern in terms of reliability, and I doubt you are the only person in the whole world who has access to critical insight onto the matter. Is it possible that your prior beliefs are unsound and unsubstantiated? > I have witnessed many incid…

Inferring one meaning for “reliability” when the original post is obviously using a different meaning suggests LLM use.

This is a class of error a human is extremely unlikely to make.

Post reply on HN