Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis is fast – I'll cache in Postgres

dizzy.zone

Post reply on HN
Print view View original

171–180 of 311 posts

Re: Redis is fast – I'll cache in Postgres

#171

Post by motorest »

odie5533 wrote: post #157
motorest wrote: post #143

Earlier quoted context omitted.

> High read/write and low-ish size. Also it's faster You posted a vague and meaningless assertion. If you do not have latency numbers and cost differences, you have absolutely nothing to show for, and you failed to provide any rationale that justified even whether any cache is required at all.

At 10k RPS you'll see a significant cost savings with Redis over DynamoDB. ElastiCache Serverless (Redis/Memcached): Typical latency is 300–500 microseconds (sub-millisecond response) DynamoDB On-Demand: Typical latency is single-digit milliseconds (usually between 1–10 milliseconds for standard requests)

> At 10k RPS you'll see a significant cost savings with Redis over DynamoDB.

You need to be more specific than that. Depending on your read/write patterns and how much memory you need to allocate to Redis, back of the napkin calculations still point to the fact that Redis can still cost >$1k/month more than DynamoDB.

Did you actually do the math on what it costs to run Redis?

Re: Redis is fast – I'll cache in Postgres

#172

Post by ralegh »

motorest wrote: post #146
lemagedurage wrote: post #139

Earlier quoted context omitted.

"If we don't need performance, we don't need caches" feels like a great broader takeaway here.

> "If we don't need performance, we don't need caches" feels like a great broader takeaway here. I don't think this holds true. Caches are used for reasons other than performance. For example, caches are used in some scenarios for stampede protection to mitigate DoS attacks. Also, the impact of caches on performance is sometimes negative. With distributed caching, each match and put require a network request. Even wh…

DoS is a performance problem, if your server was infinitely fast with infinite storage they wouldnt be an issue.

Re: Redis is fast – I'll cache in Postgres

#173

Post by motorest »

odie5533 wrote: post #130
hvb2 wrote: post #97

Earlier quoted context omitted.

You would need to get to insane read counts pretty much 24/7 for this to work out. For HA redis you need at least 6 instances, 2 regions * 3 AZs. And you're paying for all of that 24/7. And if you truly have 24/7 use then just 2 regions won't make sense as the latency to get to those regions from the other side of the globe easily removes any caching benefit.

It's $9/mo for 100 MB of ElastiCache Serverless which is HA. It's $15/mo for 2x cache.t4g.micro nodes for ElastiCache Valkey with multi-az HA and a 1-year commitment. This gives you about 400 MB. It very much depends on your use case though if you need multiple regions then I think DynamoDB might be better. I prefer Redis over DynamoDB usually because it's a widely supported standard.

> It's $9/mo for 100 MB of ElastiCache Serverless which is HA.

You need to be more specific with your scenario. Having to cache 100MB of anything is hardly a scenario that involves introducing a memory cache service such as Redis. This is well within the territory of just storing data in a dictionary. Whatever is driving the requirement for Redis in your scenario, performance and memory clearly isn't it.

Re: Redis is fast – I'll cache in Postgres

#174

Post by KronisLV »

jamesblonde wrote: post #43

Why do we promote articles like this that have nice graphs and are well written, when they should get a grade 'F' as an actual benchmark study. The way it is presented, a casual reader would think Postgres is 2/3rds the performance of Redis. Good god. He even admits Postgres maxxed out its 2 cores, but Redis was bottlenecked by the HTTP server. We need more of an academic, not a hacker, culture for benchmarks.

I feel like the outrage is unwarranted.

> The way it is presented, a casual reader would think Postgres is 2/3rds the performance of Redis.

If a reader cares about the technical choice, they'll probably at least read enough to learn of the benchmarks in this popular use case, or even just the conclusion:

> Redis is faster than postgres when it comes to caching, there’s no doubt about it. It conveniently comes with a bunch of other useful functionality that one would expect from a cache, such as TTLs. It was also bottlenecked by the hardware, my service or a combination of both and could definitely show better numbers. Surely, we should all use Redis for our caching needs then, right? Well, I think I’ll still use postgres. Almost always, my projects need a database. Not having to add another dependency comes with its own benefits. If I need my keys to expire, I’ll add a column for it, and a cron job to remove those keys from the table. As far as speed goes - 7425 requests per second is still a lot. That’s more than half a billion requests per day. All on hardware that’s 10 years old and using laptop CPUs. Not many projects will reach this scale and if they do I can just upgrade the postgres instance or if need be spin up a redis then. Having an interface for your cache so you can easily switch out the underlying store is definitely something I’ll keep doing exactly for this purpose.

I might take an issue with the first sentence (might add "...at least when it comes to my hardware and configuration."), but the rest seems largely okay.

As a casual reader, you more or less just get:

  * Oh hey, someone's experience and data points. I won't base my entire opinion upon it, but it's cool that people are sharing their experiences.
  * If I wanted to use either, I'd probably also need to look into bottlenecks, even the HTTP server, something you might not look into at first!
  * Even without putting in a lot of work into tuning, both of the solutions process a lot of data and are within an order of magnitude when it comes to performance.
  * So as a casual reader, for casual use cases, it seems like the answer is - just pick whatever feels the easiest.
If I wanted to read super serious benchmarks, I'd go looking for those (which would also have so many details that they would no longer be a casual read, short of just the abstract, but them I'm missing out on a lot anyways), or do them myself. This is more like your average pop-sci article, nothing wrong with that, unless you're looking for something else.

Eliminating the bottlenecks would be a cool followup post though!

Re: Redis is fast – I'll cache in Postgres

#175

Post by m000 »

lelanthran wrote: post #149
m000 wrote: post #125

Earlier quoted context omitted.

The use case is not representative of a real-life scenario, so the value of the presented results are minimal. A takeaway could be that you can dedicate a postgres instance for caching and have acceptable results. But who does that? Even for a relatively simple intranet app, your #1 cost when deploying in Google Cloud would probably be running Postgres. Redis OTOH is dirt cheap.

> The use case is not representative of a real-life scenario, so the value of the presented results are minimal. Maybe I'm reading the article wrong, but it is representative of any application that uses a PosgreSQL server for data, correct? In what way is that not a real-life scenario? I've deployed Single monolith + PostgreSQL to about 8 different clients in the last 2.5 years. It's my largest source of income.

When you run a relational database, you typically do it for the joins, aggregations, subqueries, etc. So a real-life scenario would include some application actually putting some stress on postgres.

If your don't mind overprovisioning your postgres, yes I guess the presented benchmarks are kind of representative. But they also don't add anything that you didn't know without reading the article.

Re: Redis is fast – I'll cache in Postgres

#176

Post by IanCal »

lemagedurage wrote: post #139
IanCal wrote: post #129

Earlier quoted context omitted.

I disagree. They found that Postgres, without tuning, was easily fast enough on low level hardware and would come with the benefit of not deploying another service. Additionally tuning it isn’t really relevant. If the defaults are fine for a use case then unless I want to tune it for personal interest it’s either a poor use of my fun time or a poor use of my clients funds.

"If we don't need performance, we don't need caches" feels like a great broader takeaway here.

A cache being fast enough doesn’t mean no caching is relevant - I’m not sure why you’d equate the two.

Re: Redis is fast – I'll cache in Postgres

#177

Post by conradfr »

ezekiel68 wrote: post #51

> Both postgres and redis are used with the out of the box settings Ugh. I know this gives the illusion of fairness, but it's not how any self-respecting software engineer should approach benchmarks. You have hardware. Perhaps you have virtualized hardware. You tune to the hardware. There simply isn't another way, if you want to be taken seriously. Some will say that in a container-orchestrated environment, tuning go…

But why doesn't Postgres tuned itself based on the system is running on, at least the basics based on available RAM & cores?

Re: Redis is fast – I'll cache in Postgres

#178

Post by vidarh »

kijin wrote: post #170
vidarh wrote: post #122

Earlier quoted context omitted.

On one hand I agree with you, but on the other hand defaults matter because I regularly see systems with the default config and no attempt to tune. Benchmarking the defaults and benchmarking a tuned setup will measure very different things, but both of them matter.

Defaults have all sorts of assumptions built into them. So if you compare different programs with their respective defaults, you are actually comparing the assumptions that the developers of those programs have in mind. For example, if you keep adding data to a Redis server under default config, it will eat up all of your RAM and suddenly stop working. Postgres won't do the same, because its default buffer size is qu…

Yes, all of this is fine but none of it address my point:

Enough people use the default settings that benchmarking the default settings is very relevant.

It often isn't a good thing to rely on the defaults, but it's nevertheless the case that many do.

(Yes, it is also relevant to benchmark tuned versions, as I also pointed out, my argument was against the claim that it is somehow unfair not to tune)

Re: Redis is fast – I'll cache in Postgres

#179

Post by scns »

motorest wrote: post #165
baobun wrote: post #155

Earlier quoted context omitted.

No need to be so combative. Take a chill pill, zoom out and look at the reliability of the entire system and its services rather than the db in isolation. If postgres has issues, it can affect the reliability of the service further if it's also running the cache. Besides, having the cache on separate hardware can reduce the impact on the db on spikes, which can also factor into reliability. Having more headroom for m…

> No need to be so combative. You're confusing being "combative" with asking you to substantiate your extraordinary claims. You opted to make some outlandish and very broad sweeping statements, and when asked to provide any degree of substance, you resorted to talk about "chill pills"? What does that say about the substance if your claims? > If postgres has issues, it can affect the reliability of the service further…

> You opted to make some outlandish and very broad sweeping statements, and when asked to provide any degree of substance, you resorted to talk about "chill pills"?

You are not answering to OP here. Maybe it's time for a little reflection?

Re: Redis is fast – I'll cache in Postgres

#180

Post by simonw »

Timshel wrote: post #132
ezekiel68 wrote: post #51

> Both postgres and redis are used with the out of the box settings Ugh. I know this gives the illusion of fairness, but it's not how any self-respecting software engineer should approach benchmarks. You have hardware. Perhaps you have virtualized hardware. You tune to the hardware. There simply isn't another way, if you want to be taken seriously. Some will say that in a container-orchestrated environment, tuning go…

> for only about five minutes of LLM query or web search I think I have more trust in the PG defaults that in the output of a LLM or copy pasting some configuration I might not really understand ...

So run the LLM in an agent loop: give it a benchmarking tool, let it edit the configuration and tell it to tweak the settings and measure and see how much if a performance improvement it can get.

That's what you'd do by hand if you were optimizing, so save some time and point Claude Code or Codex CLI or GitHub Copilot at it and see what happens.

Post reply on HN