Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis is fast – I'll cache in Postgres

dizzy.zone

Post reply on HN
Print view View original

51–60 of 311 posts

Re: Redis is fast – I'll cache in Postgres

#51

Post by ezekiel68 »

> Both postgres and redis are used with the out of the box settings

Ugh. I know this gives the illusion of fairness, but it's not how any self-respecting software engineer should approach benchmarks. You have hardware. Perhaps you have virtualized hardware. You tune to the hardware. There simply isn't another way, if you want to be taken seriously.

Some will say that in a container-orchestrated environment, tuning goes out the window since "you never know" where the orchestrator will schedule the service but this is bogus. If you've got time to write a basic deployment config for the service on the orchestrator, you've also got time to at least size the memory usage configs for PostgreSQL and/or Redis. It's just that simple.

This is the kind of thing that is "hard and tedious" for only about five minutes of LLM query or web search time and then you don't need to revisit it again (unless you decide to change the orchestrator deployment config to give the service more/less resources). It doesn't invite controversy to right-size your persistence services, especially if you are going to publish the results.

Re: Redis is fast – I'll cache in Postgres

#52

Post by motorest »

frollogaston wrote: post #33
maxbond wrote: post #31

Earlier quoted context omitted.

Expiring keys in Postgres with a created_at column and a pg_cron job is very easy (at least, if you're comfortable in Postgres). Redis is world class though of course, and can be deployed turn-key in basically any environment. If you're more comfortable in Redis than Postgres, more power to you. Different choices can be pragmatic to different people. Personally for a greenfield project, my thinking would be that I am…

I'm comfy with Postgres though, like I'll center my entire backend around it and do the heavy lifting in SQL (never ORM). It's more that I don't want to depend on a cronjob for something as fundamental as a cache. Usually Postgres costs a lot more than Redis if you're paying for a platform. Like a decent Redis or memcached in Heroku is free. And I don't want to waste precious Postgres connections or risk bogging down…

> Usually Postgres costs a lot more than Redis if you're paying for a platform.

You need to back up your unbelievable assertion with facts. Memory cache is typically far more expensive than a simple database, specially as provisioning the same memory capacity as RAM is orders of magnitude more expensive than storing the equivalent data in a database.

Re: Redis is fast – I'll cache in Postgres

#53

Post by maxbond »

frollogaston wrote: post #45
maxbond wrote: post #44

Earlier quoted context omitted.

I can understand being nervous about some cron job running on some other service, but what's concerning about a cron job managed inside of Postgres with pg_cron? If that doesn't run, your database is probably down anyway. Postgres might cost more but I'm probably already paying. I agree that exhausting connections and writing at a high rate are easy ways to bring down Postgres, but I'm personally not going to worry a…

I might be ok with this if it were built in, but pg_cron is an extension, so first off you might not even have access to it. And then you still have to monitor it.

Seems like it's available on all the major providers.

Re: Redis is fast – I'll cache in Postgres

#55

Post by motorest »

dizzyVik wrote: post #30
iamcalledrob wrote: post #3

This isn't a great test setup. It's testing RTT rather than the peak throughput of Redis. I'd suggest using Redis pipelining -- or better: using the excellent rueidis redis client which performs auto-pipelining. Wouldn't be surprising to see a 10x performance boost. https://github.com/redis/rueidis

Author here. Redis is definitely faster. I was specifically not going for absolute peak performance for either redis or postgres - that would require going down to the wire protocols. The idea was emphasize that there' a "good enough" level of performance. Once you need that sort of speed - sure, there are ways to achieve it.

> The idea was emphasize that there' a "good enough" level of performance. Once you need that sort of speed - sure, there are ways to achieve it.

To this I would add that more often than not the extra cost and complexity of a memory cache does not justify shaving off a few hypothetical milliseconds from a fetch.

On top of that, some nosql offerings from popular cloud providers already have CRUD operations faster than 20ms.

Re: Redis is fast – I'll cache in Postgres

#56

Post by whateveracct »

jamesblonde wrote: post #43

Why do we promote articles like this that have nice graphs and are well written, when they should get a grade 'F' as an actual benchmark study. The way it is presented, a casual reader would think Postgres is 2/3rds the performance of Redis. Good god. He even admits Postgres maxxed out its 2 cores, but Redis was bottlenecked by the HTTP server. We need more of an academic, not a hacker, culture for benchmarks.

most people with blogs don't know what they're doing. or don't care to know? sadly they get hired at companies and everyone does what they say cuz they have a blog. i've seen some shit in that department it's wild how much some people really are imposters.

Re: Redis is fast – I'll cache in Postgres

#57

Post by adamhartenz »

jamesblonde wrote: post #49
dizzyVik wrote: post #47

Earlier quoted context omitted.

There's a reason this is on my blog and not a paper in a journal. This isn't supposed to show the absolute speed of either tool, the benchmark is not set up for that. I do state that redis has more performance on the table in the blog post.

[flagged]

That can't have felt great having your tantrum spotlighted by the author.

Re: Redis is fast – I'll cache in Postgres

#58

Post by maxbond »

motorest wrote: post #50
frollogaston wrote: post #27

Earlier quoted context omitted.

Yeah, the article was like "I always need a DB anyway" but then sets up an extra cronjob to expire keys, plus more code. I get YAGNI and avoiding deps, but this is really extra stuff to deal with. Maybe Postgres could use a caching feature. Until then, I'm gonna drop in Redis or memcached instead of reinventing the wheel.

> Yeah, the article was like "I always need a DB anyway" but then sets up an extra cronjob to expire keys, plus more code. You do not need cron jobs to do cache. Sometimes you don't even need a TTL. All you need is a way to save data in a way that is easy and cheaper to retrieve. I feel these comments just misinterpret what a cache is by confusing it with what some specific implementation does. Perhaps that's why we…

If we don't use a TTL, aren't we going to have to either accept that our cache will grow without bounds or take an even more sophisticated approach (like tracking access times instead of creation times)? Is there something simpler I'm not seeing?

Re: Redis is fast – I'll cache in Postgres

#59

Post by nhumrich »

neoden wrote: post #46
coder543 wrote: post #36

Earlier quoted context omitted.

I keep hoping Postgres will one day have the ability to mark a timestamp column as an expiry column. It would be useful for all kinds of things beyond caching, including session tokens, feature flags, background jobs, rate limiting, delayed deletion (as a variant of soft deletion), etc. It seems like the autovacuum could take care of these expired rows during its periodic vacuum. The query planner could automatically…

One could use a trigger for this. All we need is to setup a trigger that would delete all expired records looking at some timestamp column on update. That would eat up some latency but as was said, most projects would find it good enough anyway.

I use pg cron for this. But I don't have a need for TTL to be to the minute accurate, or even to the hour.

Re: Redis is fast – I'll cache in Postgres

#60

Post by motorest »

jamesblonde wrote: post #43

Why do we promote articles like this that have nice graphs and are well written, when they should get a grade 'F' as an actual benchmark study. The way it is presented, a casual reader would think Postgres is 2/3rds the performance of Redis. Good god. He even admits Postgres maxxed out its 2 cores, but Redis was bottlenecked by the HTTP server. We need more of an academic, not a hacker, culture for benchmarks.

> He even admits Postgres maxxed out its 2 cores, but Redis was bottlenecked by the HTTP server.

What exactly is your point? That you can further optimize either option? Well yes, that comes at no suprise. I mean, the latencies alone are in the range of some transcontinental requests. Were you surprised that Redis outperformed Postgres? I hardly think so.

So what's the problem?

The main point that's proven is that there is indeed diminishing returns in terms of performance. For applications where you can afford an extra 20ms when hitting a cache, caching using a persistent database is an option. For some people, it seems this fact was very surprising. That's food for thought, isn't it?

Post reply on HN