Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis is fast – I'll cache in Postgres

dizzy.zone

Post reply on HN
Print view View original

201–210 of 311 posts

Re: Redis is fast – I'll cache in Postgres

#201

Post by lelanthran »

Implicated wrote: post #198
lelanthran wrote: post #149

Earlier quoted context omitted.

> The use case is not representative of a real-life scenario, so the value of the presented results are minimal. Maybe I'm reading the article wrong, but it is representative of any application that uses a PosgreSQL server for data, correct? In what way is that not a real-life scenario? I've deployed Single monolith + PostgreSQL to about 8 different clients in the last 2.5 years. It's my largest source of income.

> I've deployed Single monolith + PostgreSQL to about 8 different clients in the last 2.5 years. It's my largest source of income. And... do you do that with the default configuration?

> And... do you do that with the default configuration?

Yes. Internal apps/LoB apps for a large company might have, at most 5k users. PostgreSQL seems to manage it fine, none of my metrics are showing high latencies even when all employees log on in the morning during the same 30m period.

Re: Redis is fast – I'll cache in Postgres

#202

Post by Implicated »

flanked-evergl wrote: post #114
mehphp wrote: post #7

I think you just convinced me to drop redis for my new project. Definitely a premature optimization on my part.

We dropped Redis from a 4 year old app that had a rapidly growing userbase. Best choice ever. We never looked back once other than to think how annoying it was to have to deal with Redis in addition to Postgres.

Sincerely (Feel the need to add that given the tension around here in these comments), I'm curious how Redis was annoying. Can you give any detail/insight?

Re: Redis is fast – I'll cache in Postgres

#203

Post by lelanthran »

m000 wrote: post #175
lelanthran wrote: post #149

Earlier quoted context omitted.

> The use case is not representative of a real-life scenario, so the value of the presented results are minimal. Maybe I'm reading the article wrong, but it is representative of any application that uses a PosgreSQL server for data, correct? In what way is that not a real-life scenario? I've deployed Single monolith + PostgreSQL to about 8 different clients in the last 2.5 years. It's my largest source of income.

When you run a relational database, you typically do it for the joins, aggregations, subqueries, etc. So a real-life scenario would include some application actually putting some stress on postgres. If your don't mind overprovisioning your postgres, yes I guess the presented benchmarks are kind of representative. But they also don't add anything that you didn't know without reading the article.

> If your don't mind overprovisioning your postgres

Why would I mind it? I'm not using overpriced hosted PostgreSQL, after all.

Re: Redis is fast – I'll cache in Postgres

#204

Post by lillecarl »

lomase wrote: post #197
simonw wrote: post #180

Earlier quoted context omitted.

So run the LLM in an agent loop: give it a benchmarking tool, let it edit the configuration and tell it to tweak the settings and measure and see how much if a performance improvement it can get. That's what you'd do by hand if you were optimizing, so save some time and point Claude Code or Codex CLI or GitHub Copilot at it and see what happens.

How much that would cost?

They charge per token...

Re: Redis is fast – I'll cache in Postgres

#205

Post by Implicated »

lelanthran wrote: post #201
Implicated wrote: post #198

Earlier quoted context omitted.

> I've deployed Single monolith + PostgreSQL to about 8 different clients in the last 2.5 years. It's my largest source of income. And... do you do that with the default configuration?

> And... do you do that with the default configuration? Yes. Internal apps/LoB apps for a large company might have, at most 5k users. PostgreSQL seems to manage it fine, none of my metrics are showing high latencies even when all employees log on in the morning during the same 30m period.

I'm definitely getting the wrong kind of clients.

Kudos to you sir. Sincerely, I'm not hating, I'm actually jealous of the environment being that mellow.

Re: Redis is fast – I'll cache in Postgres

#206

Post by indymike »

lemagedurage wrote: post #139
IanCal wrote: post #129

Earlier quoted context omitted.

I disagree. They found that Postgres, without tuning, was easily fast enough on low level hardware and would come with the benefit of not deploying another service. Additionally tuning it isn’t really relevant. If the defaults are fine for a use case then unless I want to tune it for personal interest it’s either a poor use of my fun time or a poor use of my clients funds.

"If we don't need performance, we don't need caches" feels like a great broader takeaway here.

Sometimes, a cache is all about reducing expense: I.e, free cache query vs expensive API query.

Re: Redis is fast – I'll cache in Postgres

#207

Post by mj2718 »

Koffiepoeder wrote: post #103
mj2718 wrote: post #89

My laptop running OOTB Postgres in docker does 100k requests per second with ~5ms latency. How are you getting 20x slower?

If you look at their lab [0], it seems his NAS is separate from his kubernetes nodes. If he hasn't tuned his networking and NAS to the maximum, network storage may in fact add a LOT of delay on IOPS. Could be the difference between fractions of a millisecond vs actual milliseconds. If your DB load is mostly random reads this can really harm performance. Just hypothesizing here though, since it is not clear whether hi…

Sorry but if he’s using a setup that’s 20x worse than a regular laptop then I’m not really interested in his setup.

To be fair, I asked the question and you found the answer - lol, my bad.

Yes I agree using a nas that adds latency would reduce the TPS and explain his results. “Littles law”

Re: Redis is fast – I'll cache in Postgres

#208

Post by mj2718 »

nasretdinov wrote: post #100
mj2718 wrote: post #89

My laptop running OOTB Postgres in docker does 100k requests per second with ~5ms latency. How are you getting 20x slower?

That's what I'm struggling with too. Redis can also serve roughly 500k-1m QPS using just ~4-8 cores, so on two cores it should be about 100k-200k at least

Yep… this is what I expect as a baseline.

This is also why I rarely use redis - Postgres at 100k TPS is perfectly fine for all my use cases, including high usage apps.

Re: Redis is fast – I'll cache in Postgres

#209

Post by perrygeo »

IanCal wrote: post #129
ezekiel68 wrote: post #51

> Both postgres and redis are used with the out of the box settings Ugh. I know this gives the illusion of fairness, but it's not how any self-respecting software engineer should approach benchmarks. You have hardware. Perhaps you have virtualized hardware. You tune to the hardware. There simply isn't another way, if you want to be taken seriously. Some will say that in a container-orchestrated environment, tuning go…

I disagree. They found that Postgres, without tuning, was easily fast enough on low level hardware and would come with the benefit of not deploying another service. Additionally tuning it isn’t really relevant. If the defaults are fine for a use case then unless I want to tune it for personal interest it’s either a poor use of my fun time or a poor use of my clients funds.

The default shared memory is 128MiB, not even 1% of typical machines today. A benchmark run with these settings is effectively crippling your hardware by making sure 99% of your available memory is ignored by postgres. It's an invalid benchmark, unless redis is similarly crippled.

Re: Redis is fast – I'll cache in Postgres

#210

Post by dgfitz »

yomismoaqui wrote: post #169
rollcat wrote: post #167

Earlier quoted context omitted.

It's crazy how wildly inaccurate "top-of-the-list" LLMs are for straightforward yet slightly nuanced inquiries. I've asked ChatGPT to summarize Go build constraints, especially in the context of CPU microarchitectures (e.g. mapping "amd64.v2" to GOARCH=amd64 GOAMD64=v2). It repeatedly smashed its head on GORISCV64, claiming all sorts of nonsense such as v1, v2; then G, IMAFD, Zicsr; only arriving at rva20u64 et al un…

If you are going to ask ChatGPT some specific tidbit it's better to force it to search on the web. Remember, an LLM is a JPG of all the text of the internet.

Wait, what?

Isn't that the whole point, to ask it specific tidbits of information? Are we to ask it large, generic pontifications and claim success when we get large, generic pontifications back?

The narrative around these things changes weekly.

Post reply on HN