Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis is fast – I'll cache in Postgres

dizzy.zone

Post reply on HN
Print view View original

141–150 of 311 posts

Re: Redis is fast – I'll cache in Postgres

#141

Post by smacker »

I like using postgres for everything, it lets me simplify infrastructure. But using it as a cache is a bit concerning in terms of reliability, in my opinion.

I have witnessed many incidents when DB was considerably degrading. However, thanks to the cache in redis/memcache, a large part of the requests could still be processed with minimal increase in latency. If I were serving cache from the same DB instance, I guess, it would cause cache degradation too when there are any problems with the DB.

Re: Redis is fast – I'll cache in Postgres

#142

Post by lemagedurage »

Neikius wrote: post #111

I skimmed the article, but why is everyone always going for distributed cache? What is wrong with in-memory cache? Lowest latency, fast, easy to implement. Yeah ok, you have 30 million entries? Sure. You need to sync something over multiple nodes? Not sure I would call that a cache.

This is modern backend development. The server scales horizontally by default, nodes can be removed and added without disrupting service. With redis as cache, we can do e.g. rate limiting fast without tying a connection to a node, but also scale and deploy without impacting availability.

Re: Redis is fast – I'll cache in Postgres

#143

Post by motorest »

odie5533 wrote: post #131
motorest wrote: post #93

Earlier quoted context omitted.

> At certain scales Redis is cheaper. Can you specify in which scenario you think Redis is cheaper than caching things in, say, dynamodb.

High read/write and low-ish size. Also it's faster.

> High read/write and low-ish size. Also it's faster

You posted a vague and meaningless assertion. If you do not have latency numbers and cost differences, you have absolutely nothing to show for, and you failed to provide any rationale that justified even whether any cache is required at all.

Re: Redis is fast – I'll cache in Postgres

#144

Post by aiisthefiture »

smacker wrote: post #141

I like using postgres for everything, it lets me simplify infrastructure. But using it as a cache is a bit concerning in terms of reliability, in my opinion. I have witnessed many incidents when DB was considerably degrading. However, thanks to the cache in redis/memcache, a large part of the requests could still be processed with minimal increase in latency. If I were serving cache from the same DB instance, I guess…

Select by id is fast. If you’re using it as a cache and not doing select by id then it’s not a cache.

Re: Redis is fast – I'll cache in Postgres

#145

Post by lemagedurage »

dizzyVik wrote: post #47
jamesblonde wrote: post #43

Why do we promote articles like this that have nice graphs and are well written, when they should get a grade 'F' as an actual benchmark study. The way it is presented, a casual reader would think Postgres is 2/3rds the performance of Redis. Good god. He even admits Postgres maxxed out its 2 cores, but Redis was bottlenecked by the HTTP server. We need more of an academic, not a hacker, culture for benchmarks.

There's a reason this is on my blog and not a paper in a journal. This isn't supposed to show the absolute speed of either tool, the benchmark is not set up for that. I do state that redis has more performance on the table in the blog post.

The main issue is that a reader might mistake Redis as a 2X faster postgres. Memory is 1000X faster than disk (SSD) and with network overhead Redis can still be 100X as fast as postgres for caching workloads.

Otherwise, the article does well to show that we can get a lot of baseline performance either way. Sometimes a cache is premature optimisation.

Re: Redis is fast – I'll cache in Postgres

#146

Post by motorest »

lemagedurage wrote: post #139
IanCal wrote: post #129

Earlier quoted context omitted.

I disagree. They found that Postgres, without tuning, was easily fast enough on low level hardware and would come with the benefit of not deploying another service. Additionally tuning it isn’t really relevant. If the defaults are fine for a use case then unless I want to tune it for personal interest it’s either a poor use of my fun time or a poor use of my clients funds.

"If we don't need performance, we don't need caches" feels like a great broader takeaway here.

> "If we don't need performance, we don't need caches" feels like a great broader takeaway here.

I don't think this holds true. Caches are used for reasons other than performance. For example, caches are used in some scenarios for stampede protection to mitigate DoS attacks.

Also, the impact of caches on performance is sometimes negative. With distributed caching, each match and put require a network request. Even when those calls don't leave a data center, they do cost far more than just reading a variable from memory. I already had the displeasure of stumbling upon a few scenarios where cache was prescribed in a cargo cult way and without any data backing up the assertion, and when we took a look at traces it was evident that the bottleneck was actually the cache itself.

Re: Redis is fast – I'll cache in Postgres

#147

Post by smacker »

aiisthefiture wrote: post #144
smacker wrote: post #141

I like using postgres for everything, it lets me simplify infrastructure. But using it as a cache is a bit concerning in terms of reliability, in my opinion. I have witnessed many incidents when DB was considerably degrading. However, thanks to the cache in redis/memcache, a large part of the requests could still be processed with minimal increase in latency. If I were serving cache from the same DB instance, I guess…

Select by id is fast. If you’re using it as a cache and not doing select by id then it’s not a cache.

absolutely. But when PG is running out of open connections or has already consumed all available CPU even the simplest query will struggle.

Re: Redis is fast – I'll cache in Postgres

#148

Post by hvb2 »

ahoka wrote: post #121
hvb2 wrote: post #97

Earlier quoted context omitted.

You would need to get to insane read counts pretty much 24/7 for this to work out. For HA redis you need at least 6 instances, 2 regions * 3 AZs. And you're paying for all of that 24/7. And if you truly have 24/7 use then just 2 regions won't make sense as the latency to get to those regions from the other side of the globe easily removes any caching benefit.

A 6 node cache and caching in DynamoDB, what the hell happened to the industry? Or people just call every kind of non business-object persistence cache now?

I don't understand your comment.

If you're given the requirement of highly available, how do you not end up with at least 3 nodes? I wouldn't consider a single region to be HA but I could see that argument as being paranoid.

A cache is just a store for things that expire after a while that take load of your persistent store. It's inherently eventually consistent and supposed to help you scale reads. Whatever you use for storage is irrelevant to the concept of offloading reads

Re: Redis is fast – I'll cache in Postgres

#149

Post by lelanthran »

m000 wrote: post #125
lelanthran wrote: post #82

Earlier quoted context omitted.

I'm not seeing your point. This wouldn't get an F, purely because all the parameters are documented. Conclusions aren't incorrect either, so what's the problem?

The use case is not representative of a real-life scenario, so the value of the presented results are minimal. A takeaway could be that you can dedicate a postgres instance for caching and have acceptable results. But who does that? Even for a relatively simple intranet app, your #1 cost when deploying in Google Cloud would probably be running Postgres. Redis OTOH is dirt cheap.

> The use case is not representative of a real-life scenario, so the value of the presented results are minimal.

Maybe I'm reading the article wrong, but it is representative of any application that uses a PosgreSQL server for data, correct?

In what way is that not a real-life scenario? I've deployed Single monolith + PostgreSQL to about 8 different clients in the last 2.5 years. It's my largest source of income.

Re: Redis is fast – I'll cache in Postgres

#150

Post by phiresky »

lemagedurage wrote: post #145
dizzyVik wrote: post #47

Earlier quoted context omitted.

There's a reason this is on my blog and not a paper in a journal. This isn't supposed to show the absolute speed of either tool, the benchmark is not set up for that. I do state that redis has more performance on the table in the blog post.

The main issue is that a reader might mistake Redis as a 2X faster postgres. Memory is 1000X faster than disk (SSD) and with network overhead Redis can still be 100X as fast as postgres for caching workloads. Otherwise, the article does well to show that we can get a lot of baseline performance either way. Sometimes a cache is premature optimisation.

If your cache fits in Redis then it fits in RAM, if your cache fits in RAM then Postgres will serve it from RAM just as well.

Writes will go to RAM as well if you have synchronous=off.

Post reply on HN